Date: Sun, 05 Apr 2009 06:23:34 -0400 From: Michael Powell <nightrecon@verizon.net> To: freebsd-questions@freebsd.org Subject: Re: how to recover after power outage Message-ID: <gra0pd$tph$1@ger.gmane.org> References: <AF401DB4-C4D1-44B0-B9D2-A34D81596481@identry.com> <4ad871310904050141k11eda14vd8db9224f4384757@mail.gmail.com> <1ED100F1-51AE-478D-873C-40FF43EA17FB@identry.com>
next in thread | previous in thread | raw e-mail | index | archive | help
John Almberg wrote: > > On Apr 5, 2009, at 4:41 AM, Glen Barber wrote: > >> On Sun, Apr 5, 2009 at 2:59 AM, John Almberg <jalmberg@identry.com> >> wrote: >>> Blast... my beautiful FreeBSD servers were rudely switched off >>> when my data >>> had a power outage a couple hours ago. They restored power about >>> 30 minutes >>> later, and one box came up no problem. >>> >>> The other has a login prompt on the serial console, but my login >>> does not >>> work. I get a "Login incorrect" message, even though the username/ >>> password >>> is correct. >>> >> >> Can you log in as *any* user? Even root login fails? > > Can't log in at all. > >> >>> When I try to SSH into the box, I get this (server name changed): >>> >>> $ ssh user@example.com -p 48420 >>> ssh: connect to host example.com port 48420: Connection refused >>> >>> In other words, I seem to be locked out. >>> >>> I don't want to do anything drastic without having a good idea >>> what I'm >>> doing. Any suggestions, much appreciated. >>> >> >> What was the previous (estimated) uptime on the machine? > > Several months, at least. > >> In other >> words, did you change something and not/forget to restart the service? > > I don't believe so, but if I forgot it, then I guess anything is > possible. > >> Have you tried ssh-ing to port 22 to see if the setting was changed >> back to default? > > I can't at the moment, because the guys at NYI are working on the > box. They have run fsck, which doesn't seem to have solved the problem. > >> >> Are there any other services on this box? If so, are they running? > > The main app is MySQL. I don't think it is running, but can't really > tell unless I can log in. > > I have backups, and while NYI is trying to get this box running, I'm > setting up a new database server, just in case... > If you were lucky having the guys at NYI login to single user mode at the console and run fsck in an attempt to clear up minor file system damage would have squared things away. MySQL is not real happy if there has been fs damage to the underlying files and their .bin logs. However, not being able to log in to a basic service like SSH is not good. Whether or not MySQL wants to come up SSH should still be working. In the end the guys at NYI are probably going to have to do a full system load and restore the last backup, and/or replace defective hardware. I have seen old hard drives in RAID arrays that had perked along for years show no hint of any problem. Power down the machine to do something like blow the dust out or stick in some more memory sticks and it won't come up again. Had I not powered down it may have happily run a while longer. I have seen drives fail like this before, especially when they are fairly old. At this stage you can only emit SIGH and replace/rebuild. But if the NYI guys are responsible for providing you with a running system the onus is on them to get it going again, at least up to a certain point. After that you would need to pick up and carry the ball the rest of the way. -Mike
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?gra0pd$tph$1>