From owner-freebsd-ports@FreeBSD.ORG Wed May 13 06:46:34 2015 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB405D68 for ; Wed, 13 May 2015 06:46:34 +0000 (UTC) Received: from udns.ultimatedns.net (unknown [IPv6:2602:d1:b4d6:e600:4261:86ff:fef6:aa2a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AB10B1553 for ; Wed, 13 May 2015 06:46:34 +0000 (UTC) Received: from ultimatedns.net (localhost [127.0.0.1]) by udns.ultimatedns.net (8.14.9/8.14.9) with ESMTP id t4D6kupB073162; Tue, 12 May 2015 23:47:02 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) To: freebsd-ports@freebsd.org, Scott Bennett In-Reply-To: <201505130512.t4D5Cpqa011080@sdf.org> References: <201505120617.t4C6HkA3019096@sdf.org> <0dcbdcb9fdfdebc53a2563b11494f61b@ultimatedns.net>, <201505130512.t4D5Cpqa011080@sdf.org> From: "Chris H" Subject: Re: damage to pkg's sqlite data base Date: Tue, 12 May 2015 23:47:02 -0700 Content-Type: text/plain; charset=UTF-8; format=fixed MIME-Version: 1.0 Message-id: <5f1749d3e664fbb487aa98f160e6cf92@ultimatedns.net> Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 May 2015 06:46:34 -0000 On Wed, 13 May 2015 00:12:51 -0500 Scott Bennett wrote > "Chris H" wrote: > > > On Tue, 12 May 2015 01:17:46 -0500 Scott Bennett wrote > > > > > For nearly two weeks I've been stymied by an apparently damaged record > > > in the sqlite data base used by pkg(8) and pkg-static(8). Unfortunately, > > > it is a record for a port that is depended upon rather heavily, lang/gcc. > > > lang/gcc compiled and linked just fine, but any attempt to install the > > > result ends up like this. > > > > > > ===> Checking if gcc already installed > > > ===> Registering installation for gcc-4.8.4_3 > > > Installing gcc-4.8.4_3... > > > pkg-static: sqlite error while executing iterator in file > > > pkgdb_iterator.c:931: database disk image is malformed pkg-static: sqlite > > > error while executing INSERT OR REPLACE INTO files (path, sha256, > > > package_id) VALUES (?1, ?2, ?3) in file pkgdb.c:1722: database disk image > > > is malformed *** Error code 70 > > > > > > Stop. > > > make: stopped in /usr/ports/lang/gcc > > > > > > I'm not familiar with sqlite commands, nor with the structure of the > > > pkg data base, nor any of several other relevant topics. But I do have > > > questions: > > > > > > 1) how might the damage have happened? > > > > > > 2) how might such an event be avoided/prevented in the future? > > > > > > 3) most urgently, how can I *fix* it? > > You should get a backup made on the nightly cron(8) (periodic(8)). > > It'll be located in /var/backups > > Off the top of my head I can't remember which of the 2, is which. > > But a simple size comparison should tell the tale. :) > > The problem began much farther back than two days ago, but I should > be able to restore it from two weeks or more in the past and then find > the most recent version that is not corrupted. > > > Simply rename your (now) corrupt db, and copy the backup over. > > However, if I do that, then what happens to all the ports that have > been updated or added since that version of the data base was backed up? > I have run "portmaster -a" (with some additional options) quite a few > times since the lang/gcc problem first appeared, so an old local.sqlite > will no longer accurately reflect what is currently installed. > > > > I ran into this problem as well, and this was how I was finally > > able to overcome it. > > > > Hope this helps! > > > > --Chris > > > > > > > > 4) I was unable to find any instructions for recreating a pkg data > > > base if the data base gets damaged/destroyed. Is there a way to > > > do that that I missed? > > There must be a way to do this, right? I mean, really, it's pretty > fundamental that no new data base be put into production without a way to > rebuild it. The FreeBSD developers haven't really broken so ancient and > basic a principle, have they? So what's the trick? What is the method > to rebuild /var/db/pkg/local.sqlite from scratch based upon the currently > installed ports/packages? I whined about it the first time my DB blew up. It's become corrupted several times since on different boxes/versions. *but* after the first time, I made it a habit of making a copy of it *before* embarking on an upgrade, or install of any ports. Seems it's the only way to save yourself. pkg(8), to the best of my knowledge doesn't perform a backup prior to any of this. So I *do*. Annoying, but it beats attempting to recover from a failed port upgrade/install. > > > Scott Bennett, Comm. ASMELG, CFIAG > ********************************************************************** > * Internet: bennett at sdf.org *xor* bennett at freeshell.org * > *--------------------------------------------------------------------* > * "A well regulated and disciplined militia, is at all times a good * > * objection to the introduction of that bane of all free governments * > * -- a standing army." * > * -- Gov. John Hancock, New York Journal, 28 January 1790 * > ********************************************************************** > _______________________________________________ > freebsd-ports@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-ports > To unsubscribe, send any mail to "freebsd-ports-unsubscribe@freebsd.org" --Chris --