Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 18 Nov 2000 22:10:57 -0800
From:      "Dave Walton" <dwalton@acm.org>
To:        fs@freebsd.org
Subject:   corrupted filesystem
Message-ID:  <3A16FE71.20481.69FB69@localhost>

next in thread | raw e-mail | index | archive | help
Boy have I got a stinker of a mess.  A server crashed, apparently 
due to a power supply problem, and messed up its filesystem 
pretty thoroughly.  I need some help digging through the piles of 
bits.  (As an aside, I'd had the notion that only open files were at 
risk for corruption.  These problems seem more widespread than 
that.  Why might that be?   Would softupdates have prevented the 
problem?)

Anyway, I ran fsck -n to have a peek, and it ran for a couple of 
hours before I killed it, spewing an incredible number of problem 
reports.  I am reluctant to use fsck -y for fear that it would cause 
even more data loss, and answering all those questions manually 
is right out.  This is an 85GB filesystem.  It'd take forever, even if I 
did know the "right" answers.

What I'd really like to do is just rescue as much as I can from 
/usr/home, then clean up the rest of the mess with newfs.  To that 
end, I mounted the /usr filesystem read only on /mnt and started 
poking around.  Right off the bat I ran into this:

------------------------------------------------------------
/mnt# ls -alF
ls: libdata: Bad file descriptor
ls: local: Bad file descriptor
ls: share: Bad file descriptor
ls: src: Bad file descriptor
total 1235043903
drwxr-xr-x     17 root       wheel                       512 Aug 24 23:45 ./
drwxr-xr-x     15 root       wheel                       512 Nov 18 02:29 ../
drwxr-xr-x      2 root       wheel                      6656 Aug 25 06:07 
bin/
drwxr-xr-x      3 root       wheel                       512 Aug 23 19:17 
compat/
drwxr-xr-x      3 root       wheel                      1024 Aug 23 18:26 
games/
-rws-w--wx  32458 389421524  3725029886  
7966077931114349813 Jan 30  2022 home*
drwxr-xr-x     35 root       wheel                      3072 Aug 24 05:55 
include/
drwxr-xr-x      4 root       wheel                      6656 Aug 24 05:57 
lib/
drwxr-xr-x      8 root       wheel                      1024 Aug 24 05:57 
libexec/
drwxr-xr-x      3 root       wheel                       512 Aug 24 23:57 
obj/
drwxr-xr-x     51 root       wheel                      1024 Aug 23 18:47 
ports/
-rw-r-----      1 root       operator                2097120 Nov 17 04:23 
quota.group
-rw-r-----      1 root       operator                2097120 Nov 17 04:23 
quota.user
drwxr-xr-x      2 root       wheel                      4096 Aug 25 06:07 
sbin/
drwxrwxrwt      3 root       wheel                      1536 Nov 16 10:08 
tmp/
/mnt#
------------------------------------------------------------

By some magic, home has become a regular file instead of a dir!
Well, that won't do at all, because that is the place I most need to 
get to.  So I gritted my teeth and fired up fsdb (first time ever) to 
have a look-see:

------------------------------------------------------------
/# fsdb -r /dev/rda0s1f
** /dev/rda0s1f (NO WRITE)
Examining file system `/dev/rda0s1f'
Last Mounted on /usr
current inode: directory
I=2 MODE=40755 SIZE=512
        MTIME=Aug 24 23:45:30 2000 [0 nsec]
        CTIME=Aug 24 23:45:30 2000 [0 nsec]
        ATIME=Nov 17 00:25:04 2000 [0 nsec]
OWNER=root GRP=wheel LINKCNT=17 FLAGS=0 BLKCNT=2 
GEN=6f7bd28
fsdb (inum: 2)> ls
slot 0 ino 2 reclen 12: directory, `.'
slot 1 ino 2 reclen 12: directory, `..'
slot 2 ino 7936 reclen 12: directory, `bin'
slot 3 ino 15872 reclen 16: directory, `include'
slot 4 ino 349184 reclen 12: directory, `lib'
slot 5 ino 380928 reclen 16: directory, `libdata'
slot 6 ino 1190400 reclen 16: directory, `libexec'
slot 7 ino 1253888 reclen 16: directory, `local'
slot 8 ino 1261824 reclen 16: directory, `sbin'
slot 9 ino 1269760 reclen 16: directory, `share'
slot 10 ino 3222016 reclen 12: directory, `src'
slot 11 ino 4063232 reclen 16: directory, `games'
slot 12 ino 1222148 reclen 16: directory, `ports'
slot 13 ino 15014935 reclen 16: directory, `compat'
slot 14 ino 15038738 reclen 12: directory, `obj'
slot 15 ino 15443479 reclen 12: directory, `tmp'
slot 16 ino 3039518 reclen 16: directory, `home'
slot 17 ino 27 reclen 20: regular, `quota.user'
slot 18 ino 28 reclen 248: regular, `quota.group'
fsdb (inum: 2)> cd home
component `home': current inode: regular file
I=3039518 MODE=104723 SIZE=7966077931114349813
        MTIME=Jan 30 19:02:03 2022 [2140389655 nsec]
        CTIME=Nov  2 18:02:35 1975 [-556098662 nsec]
        ATIME=Feb  7 04:40:01 2016 [374691499 nsec]
OWNUID=389421524 GID=3725029886 LINKCNT=32458 
FLAGS=0xb03ec3e0 BLKCNT=933a8bb1 GEN=7f060afc
fsdb (inum: 3039518)> 
------------------------------------------------------------

Ok, now I'm really confused!  First fsdb reports home as a 
directory, but when I cd into it, it becomes a regular file?  How can 
that be?

So that's where I am.  Any suggestions for getting into home are 
most welcome.  I have the feeling that home itself may be a lost 
cause.  How can I search for all directories that had home as a 
parent and relink them somewhere else?  Any other ideas?

In summary....  HELP!  :)

Thanks for listening,
Dave



----------------------------------------------------------------------
Dave Walton                                            dwalton@acm.org
----------------------------------------------------------------------


To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-fs" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3A16FE71.20481.69FB69>