From owner-freebsd-questions@FreeBSD.ORG Sat Feb 11 22:28:05 2012 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7BAC21065676 for ; Sat, 11 Feb 2012 22:28:05 +0000 (UTC) (envelope-from levitch@iglou.com) Received: from rdsmtp.iglou.com (rdsmtp.iglou.com [192.107.41.63]) by mx1.freebsd.org (Postfix) with ESMTP id 363268FC15 for ; Sat, 11 Feb 2012 22:28:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=iglou.com; s=alpha; h=Content-Type:MIME-Version:References:Message-ID:In-Reply-To:Subject:To:From:Date; bh=mOJvmRLMdTHo6wF8qV5JWHjy83j1lPq7hmdSjHLkpMA=; b=guAU8HR0kjiZUTITsca0jniqBJy9vryOiRiQjrlt96SQP8zCvDfkK0ScpWB3HpWWUq0IwgLZWrYcxPxcB3ltULzwsO5O7i/541j6e+gyrhux8ePw7L7rpu6I/EM53JQoU3b4fNUr56XF89wS/9EZgqtmihbaDqd6PW4iOqt7+yU=; Received: from iglou3.iglou.com ([192.107.41.6]:37053 helo=mail.iglou.com) by rdsmtp.iglou.com with esmtpa (Exim MTA/8.19.3) (envelope-from ) id 1RwLQK-0000hf-LR by authid with igloumta_auth for freebsd-questions@freebsd.org; Sat, 11 Feb 2012 17:28:04 -0500 Received: from shell1.iglou.com ([192.107.41.17]:33176 helo=shell1) by mail.iglou.com with esmtps (TLS cipher TLSv1:AES256-SHA:256) (Exim MTA/8.19.3) (envelope-from ) id 1RwLQK-00016V-CY for freebsd-questions@freebsd.org; Sat, 11 Feb 2012 17:28:04 -0500 Date: Sat, 11 Feb 2012 17:28:04 -0500 (EST) From: Darrel X-X-Sender: levitch@shell1 To: freebsd-questions@freebsd.org In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (GSO 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Originating-IP: 192.107.41.17 X-IgLou-Customer: 3cb6f76205bd20f518810676a67a982b Subject: update, [solved]: journal timestamp X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Feb 2012 22:28:05 -0000 > >> An amd64 running fbsd9-RC1 was shutdown overnight from the 'shutdown >> -p now' command. It reported an unclean shutdown and I ran 'fsck >> -y'. Still it will not boot and the message is "Journal timestamp >> does not match fs mount time". >> >> This is occurring for both /var and /usr. >> > > Sporadic episodes of dd, mount, fsck and the like produced no results- > apparently if the journal is out of sync then FreeBSD offers no utility to > fix it. > > With a new disk and install of FreeBSD9 then I could mount ufs /usr read-only > and copy files to the new installation and then with 'zfs list' created a > mount-point for the zfs disk and copied those files as well. > Seems like one of the disks was failing. I decided to use a partition of the large disk; i.e., ada0p9: # zpool attach bigD /dev/gpt/bigD0 /dev/ada0p9 ********************************************************** The first 74g disk is reporting errors: (146) @ 16:10:30> zpool status bigD pool: bigD state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Sat Feb 11 15:59:48 2012 3.82G scanned out of 4.19G at 5.60M/s, 0h1m to go 3.82G resilvered, 91.02% done config: NAME STATE READ WRITE CKSUM bigD ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/bigD0 ONLINE 0 0 1 (resilvering) gpt/bigD1 ONLINE 0 0 0 ada0p9 ONLINE 0 0 0 (resilvering) errors: No known data errors ********************************************************** No data errors, now spread across three disks: (149) @ 16:14:18> zpool status bigD pool: bigD state: ONLINE status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'. scan: resilvered 4.20G in 0h14m with 0 errors on Sat Feb 11 16:14:22 config: NAME STATE READ WRITE CKSUM bigD ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/bigD0 ONLINE 0 0 1 gpt/bigD1 ONLINE 0 0 0 ada0p9 ONLINE 0 0 0 errors: No known data errors ********************************************************** Just in case: (31) @ 16:16:54> gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot \ -i 1 ada0 bootcode written to ada0 ********************************************************** Clear the error, so I can run 'zpool status' again: (151) @ 16:20:21> zpool clear bigD /dev/gpt/bigD0 ********************************************************** (152) @ 16:21:00> zpool status bigD pool: bigD state: ONLINE scan: resilvered 4.20G in 0h14m with 0 errors config: NAME STATE READ WRITE CKSUM bigD ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/bigD0 ONLINE 0 0 0 gpt/bigD1 ONLINE 0 0 0 ada0p9 ONLINE 0 0 0 errors: No known data errors ********************************************************** # shutdown -r now ********************************************************** (29) @ 16:26:10> zpool scrub bigD ********************************************************** The disk is reporting more errors: (30) @ 16:27:03> zpool status bigD pool: bigD state: ONLINE status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected. action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'. scan: scrub in progress since Sat Feb 11 16:26:51 2012 1.16G scanned out of 4.19G at 3.46M/s, 0h14m to go 120K repaired, 27.76% done config: NAME STATE READ WRITE CKSUM bigD ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/bigD0 ONLINE 0 0 19 (repairing) gpt/bigD1 ONLINE 0 0 0 ada0p9 ONLINE 0 0 0 errors: No known data errors ********************************************************** If the disk fails, the data mirror should still be okay: A mirror of two or more devices. Data is replicated in an identical fashion across all components of a mirror. A mirror with N disks of size X can hold X bytes and can withstand (N-1) devices failing before data integrity is compromised. ********************************************************** (33) @ 16:41:14> zpool status bigD pool: bigD state: ONLINE status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected. action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'. scan: scrub repaired 638K in 0h14m with 0 errors on Sat Feb 11 16:41:29 2012 config: NAME STATE READ WRITE CKSUM bigD ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/bigD0 ONLINE 0 0 44 gpt/bigD1 ONLINE 0 0 0 ada0p9 ONLINE 0 0 0 errors: No known data errors ********************************************************** Darrel