Date: Mon, 23 Mar 2015 02:00:33 -0800 From: "CK" <nibbana@gmx.us> To: <freebsd-questions@freebsd.org> Subject: Re: thrashing + lost files Message-ID: <0Lx87b-1ZbIFT2r97-016gJU@mail.gmx.com>
next in thread | raw e-mail | index | archive | help
> >> On Tue, 17 Mar 2015 23:56:25 -0800, "CK" <nibbana@gmx.us> said:
>
> > I would like any thoughts or ideas on how to prevent the following
> > problem, because it is making my computer completely unusable, wasting
> > many efforts. The result is the loss of many critical files from a hard
> > drive, as if a "rm *" was done in the home directory.
>
> I don't know your hardware or RAM situation, but are you in a position
> to use ZFS? I've been using it for quite some time on a Solaris
> box, and there's nothing better -- it handles bit-rot, compression,
> and encryption automatically if you like. The only downside: it's a
> memory hog, and you MUST use ECC or risk data corruption because ZFS
> completely trusts what's in RAM.
No, I studied ZFS, but it had hardware requirements that were beyond my
hardware. I can't remember exactly, but it required a very large amount of
RAM or a very large terabyte-type hard drive. I only use 40GB, and my spare
is 60GB.
> > One bad problem under these conditions, where one has to umount a
> > partition, is that umount-ing is not possible if the partition is in
> > use, and it seems there is no easy way to find every process using a
> > partition ... is there?
>
> I seem to recall that "fuser -uc /filesystem" would do this, but I
> don't have a BSD box available to check.
Oh great, thanks! I'll study this more ...
fuser -- list IDs of all processes that have one or more files open
The fuser utility appeared in FreeBSD 9.0.
The command: ``fuser -fu .'' writes to standard output the process IDs of
processes that are using the current directory and writes to stderr an
indication of how those processes are using the directory and user names
associated with the processes that are using this directory.
> > A lost+found directory is never created, which ignorantly, I think
> > should be created and filled with the "unreferenced inodes" (if that is
> > the correct term).
>
> I know fsck is supposed to make that directory for you, but I still
> create one whenever I make a new filesystem. You never know.
>
> I've had to grub around in a lost+found directory wondering where all
> that crap is supposed to go; it's easier if you have something like a
> file-DB around to help. There's a script attached that's a shorter
> version of one I use to track my own files. It tracks most of the
> metadata (including inodes) for a directory tree.
>
> > Yes, I do [backups] of course, with a USB SDRAM device. But I still
> > lose days of work, because I can't back up every minute.
>
> Do you have another unused (preferably external) drive that could be
> added/attached to your system? I got in the habit of backing stuff
> up hourly, and it's surprisingly easy to do from cron:
Yes, the small 4GB hard drive with checksum errors and the 32GB SDRAM device
are used for backups. The checksum errors found with smartctl on the 4GB are
not relevant. The drive had problems when it was first purchased, but is
reliable in it's use as just a backup drive.
> * create a timestamp file (/usr/local/etc/TIME),
> * create a year-month-day directory elsewhere, and
> * run something like
> cd /
> find ./your/home -newer /usr/local/etc/TIME -print |
> pax -rwd -pe /other/drive/YYYY/MMDD/HHMM
> * touch /usr/local/etc/TIME
>
> You wouldn't believe how often that's gotten me out of jams. Copying
> to a local drive is fine as long as you rsync it someplace safe; what
> matters is that it's around somewhere to get you out of trouble.
Great, thanks! I never got into 'pax/cpio'. When I had time and energy for
trying to learn all things, it was something I skipped. I always used a script
that txz's a directory, and the reverse as needed. Yes, this thought occurred
to me a few days ago, and I am very glad you provided a working example of how
to use it well. It is a good idea, but it still would be preferential for
this situation never to occur.
> > I have used FreeBSD for 20 years, since 1995, and I never had problems
> > like this before.
>
> I've used FreeBSD as a Samba server for several hundred customers in
> a production environment where uptime was measured in years, and I would
> be surprised (and disgusted) if something like this ever happened to me.
>
> Good luck!
Yes, I agree. The only time I reboot is when the power blinks due to a
neighbors welding machine :) I stress my machine quite a bit with encrypting
100s of MB, 600MB MP3 resampling, many filesystem scripts, and other things.
And as I said previously, I was doing 'ls' and watching files this last time,
and after X was down, top/ps showed Opera was the only thing running, as it
was thrashing my hard drive, I could see the files vanishing from my home
directory, even Opera's own core. So, the problem definitely occurs at that
time.
Not this last time (I still have not rebooted since), but the first time this
happened, and an additional reason I turned off journaling/soft-updates, is
that when I did replace some files from backup copies, after rebooting,
fscking/mounting, the files would vanish again! hmmm. I forgot to say that
before. I should test this now, and reboot, and see if any files vanish now
that journaling/soft-updates are turned off. It seemed to have fixed the
problem the 1st time, where I had to reboot several times to make sure files
would stop vanishing on me.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0Lx87b-1ZbIFT2r97-016gJU>