From owner-freebsd-questions@FreeBSD.ORG Wed Feb 18 08:35:43 2004 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4D73F16A4CE for ; Wed, 18 Feb 2004 08:35:43 -0800 (PST) Received: from mail.watkinscontracting.com (65-119-16-245.dsl.tfb.com [65.119.16.245]) by mx1.FreeBSD.org (Postfix) with ESMTP id E525F43D1D for ; Wed, 18 Feb 2004 08:35:42 -0800 (PST) (envelope-from noonans@watkinscontracting.com) Received: from watwks04 (wat-wks-04.watkinscontracting.com [192.168.100.103]) i1IGZgOO020132 for ; Wed, 18 Feb 2004 08:35:42 -0800 (PST) (envelope-from noonans@watkinscontracting.com) From: "Sean Noonan" To: Date: Wed, 18 Feb 2004 08:35:40 -0800 Message-ID: <012801c3f63d$3fbbdc20$6764a8c0@watkinscontracting.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------=_1077122142-331-594" X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook CWS, Build 9.0.6604 (9.0.2911.0) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 In-Reply-To: <000001c3f4d9$5fdcc3d0$6706a8c0@FLOYD> Importance: Normal X-Scanned-By: MIMEDefang 2.38 Subject: SOLVED! : Please help - URGENT - disk/fsck problems! X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Feb 2004 16:35:43 -0000 This is a multi-part message in MIME format... ------------=_1077122142-331-594 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline > I'm in need of urgent help. I have a production 4.9-RELEASE > server. I track STABLE, last cvsup/make world was about two > months ago. After powering on after an extended power outage > (was given advanced notice by electric company and I shut the > server down before the outage occurred), the server hung with > disk problems and suggested I run fsck. I ran fsck and came > up with thousands of errors. I re-ran fsck and came up with > less. I repeated this process for what seems like a hundred > times. No matter what I do, though, I still have errors that > fsck seems unable to repair. HELP!! Well, seeing as that I didn't get a single response to my URGENT plea for help, and noting that necessity is the mother of all invention, I managed to solve this problem myself (48 hours after the fact with users screaming at me for 6 of them). I'm repling to my own post in the hopes it'll be archived and help some poor SOB like myself should they encounter the same problem. I don't know if it was the *right* way to solve the problem, or even if there were alternative ways. This method did work, though. It resulted in some data loss, but I was illing to live with that since I have good tape backups to restore from. Here's how I fixed my problem. After RTFM on fsck (man fsck) for the 4,341st time, I noticed under the "see also" section a utility called fsdb, the FFS debugging/editing tool. While messing with inodes and such as always scared the hell out of me, I figured I had nothing to loose at this point. I went back to my fsck output and noted that certain errors seemed to never go away, like this one: INCORRECT BLOCK COUNT I=447212 (5280 should be 4064) I assumed that the "I" above was an Inode number. I fired up fsdb with the disk slice in question as a paramenter, like this: fsdb /dev/da0s1g This gave me a "fsdb >" prompt. >From there I issued the command: clri i-number (e.g., clri 447212). This apparently nukes the Inode number given to it. I did this for all the Inode numbers that continually re-appeared after numerous fsck's. I typed "quit" (or was it "exit"?) to leave fsdb and then re-ran fsck. Problem sovled. Restored from tape (love AMANDA!) and I was good-to-go! Hopes this helps somebody... Thanks, --Sean Noonan ***************************************************************************************************************************** No employee or agent is authorized to conclude any binding agreement on behalf of Watkins Contracting, L.P., with another party by email without express written confirmation by an Officer of the company. In addition, any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Watkins Contracting, L.P. Employees of Watkins Contracting, L.P., are expressly required not to make defamatory statements and not to infringe or authorize any infringement of copyright or any other legal right by email communications. Any such communication is contrary to company policy and outside the scope of the employment of the individual concerned. The company will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising. ------------=_1077122142-331-594--