From owner-freebsd-fs Sat Nov 18 22:11:17 2000 Delivered-To: freebsd-fs@freebsd.org Received: from modgud.nordicrecords.com (h21-168-107.nordicdms.com [207.21.168.107]) by hub.freebsd.org (Postfix) with SMTP id 6B2E737B479 for ; Sat, 18 Nov 2000 22:11:12 -0800 (PST) Received: (qmail 5711 invoked by alias); 19 Nov 2000 06:11:11 -0000 Received: (qmail 5704 invoked from network); 19 Nov 2000 06:11:10 -0000 Received: from adsl-216-103-90-137.dsl.snfc21.pacbell.net (HELO thinkpad770z) (216.103.90.137) by mail.nordicrecords.com with SMTP; 19 Nov 2000 06:11:10 -0000 From: "Dave Walton" To: fs@freebsd.org Date: Sat, 18 Nov 2000 22:10:57 -0800 MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Subject: corrupted filesystem Reply-To: dwalton@acm.org Message-ID: <3A16FE71.20481.69FB69@localhost> X-mailer: Pegasus Mail for Win32 (v3.12c) Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Boy have I got a stinker of a mess. A server crashed, apparently due to a power supply problem, and messed up its filesystem pretty thoroughly. I need some help digging through the piles of bits. (As an aside, I'd had the notion that only open files were at risk for corruption. These problems seem more widespread than that. Why might that be? Would softupdates have prevented the problem?) Anyway, I ran fsck -n to have a peek, and it ran for a couple of hours before I killed it, spewing an incredible number of problem reports. I am reluctant to use fsck -y for fear that it would cause even more data loss, and answering all those questions manually is right out. This is an 85GB filesystem. It'd take forever, even if I did know the "right" answers. What I'd really like to do is just rescue as much as I can from /usr/home, then clean up the rest of the mess with newfs. To that end, I mounted the /usr filesystem read only on /mnt and started poking around. Right off the bat I ran into this: ------------------------------------------------------------ /mnt# ls -alF ls: libdata: Bad file descriptor ls: local: Bad file descriptor ls: share: Bad file descriptor ls: src: Bad file descriptor total 1235043903 drwxr-xr-x 17 root wheel 512 Aug 24 23:45 ./ drwxr-xr-x 15 root wheel 512 Nov 18 02:29 ../ drwxr-xr-x 2 root wheel 6656 Aug 25 06:07 bin/ drwxr-xr-x 3 root wheel 512 Aug 23 19:17 compat/ drwxr-xr-x 3 root wheel 1024 Aug 23 18:26 games/ -rws-w--wx 32458 389421524 3725029886 7966077931114349813 Jan 30 2022 home* drwxr-xr-x 35 root wheel 3072 Aug 24 05:55 include/ drwxr-xr-x 4 root wheel 6656 Aug 24 05:57 lib/ drwxr-xr-x 8 root wheel 1024 Aug 24 05:57 libexec/ drwxr-xr-x 3 root wheel 512 Aug 24 23:57 obj/ drwxr-xr-x 51 root wheel 1024 Aug 23 18:47 ports/ -rw-r----- 1 root operator 2097120 Nov 17 04:23 quota.group -rw-r----- 1 root operator 2097120 Nov 17 04:23 quota.user drwxr-xr-x 2 root wheel 4096 Aug 25 06:07 sbin/ drwxrwxrwt 3 root wheel 1536 Nov 16 10:08 tmp/ /mnt# ------------------------------------------------------------ By some magic, home has become a regular file instead of a dir! Well, that won't do at all, because that is the place I most need to get to. So I gritted my teeth and fired up fsdb (first time ever) to have a look-see: ------------------------------------------------------------ /# fsdb -r /dev/rda0s1f ** /dev/rda0s1f (NO WRITE) Examining file system `/dev/rda0s1f' Last Mounted on /usr current inode: directory I=2 MODE=40755 SIZE=512 MTIME=Aug 24 23:45:30 2000 [0 nsec] CTIME=Aug 24 23:45:30 2000 [0 nsec] ATIME=Nov 17 00:25:04 2000 [0 nsec] OWNER=root GRP=wheel LINKCNT=17 FLAGS=0 BLKCNT=2 GEN=6f7bd28 fsdb (inum: 2)> ls slot 0 ino 2 reclen 12: directory, `.' slot 1 ino 2 reclen 12: directory, `..' slot 2 ino 7936 reclen 12: directory, `bin' slot 3 ino 15872 reclen 16: directory, `include' slot 4 ino 349184 reclen 12: directory, `lib' slot 5 ino 380928 reclen 16: directory, `libdata' slot 6 ino 1190400 reclen 16: directory, `libexec' slot 7 ino 1253888 reclen 16: directory, `local' slot 8 ino 1261824 reclen 16: directory, `sbin' slot 9 ino 1269760 reclen 16: directory, `share' slot 10 ino 3222016 reclen 12: directory, `src' slot 11 ino 4063232 reclen 16: directory, `games' slot 12 ino 1222148 reclen 16: directory, `ports' slot 13 ino 15014935 reclen 16: directory, `compat' slot 14 ino 15038738 reclen 12: directory, `obj' slot 15 ino 15443479 reclen 12: directory, `tmp' slot 16 ino 3039518 reclen 16: directory, `home' slot 17 ino 27 reclen 20: regular, `quota.user' slot 18 ino 28 reclen 248: regular, `quota.group' fsdb (inum: 2)> cd home component `home': current inode: regular file I=3039518 MODE=104723 SIZE=7966077931114349813 MTIME=Jan 30 19:02:03 2022 [2140389655 nsec] CTIME=Nov 2 18:02:35 1975 [-556098662 nsec] ATIME=Feb 7 04:40:01 2016 [374691499 nsec] OWNUID=389421524 GID=3725029886 LINKCNT=32458 FLAGS=0xb03ec3e0 BLKCNT=933a8bb1 GEN=7f060afc fsdb (inum: 3039518)> ------------------------------------------------------------ Ok, now I'm really confused! First fsdb reports home as a directory, but when I cd into it, it becomes a regular file? How can that be? So that's where I am. Any suggestions for getting into home are most welcome. I have the feeling that home itself may be a lost cause. How can I search for all directories that had home as a parent and relink them somewhere else? Any other ideas? In summary.... HELP! :) Thanks for listening, Dave ---------------------------------------------------------------------- Dave Walton dwalton@acm.org ---------------------------------------------------------------------- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message