From owner-freebsd-ports@FreeBSD.ORG Fri Aug 20 12:10:03 2010 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD35A10656AB; Fri, 20 Aug 2010 12:10:02 +0000 (UTC) (envelope-from julien.laffaye@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5A8E58FC2C; Fri, 20 Aug 2010 12:10:01 +0000 (UTC) Received: by wyj26 with SMTP id 26so4108960wyj.13 for ; Fri, 20 Aug 2010 05:10:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=9SOmY1dcQuBdpirJOpzFs6xC1gUjorEfa7AgCboupl4=; b=rsFjPjQcKQm2mkZXLrXh/h5QgC5fdPp7yKnBenXuyR1hJZt2M4ouZgAcqHVNiZIc+e Ge6L29oOqZRkdxvJM77fIUd+NPA6ROql9VsZtFRDjsyh9DRxBqTxCVL0M3iftmDx0tZS jUrf+8Kzeje2xYdYn6r6mN9bSWHbzFqkPTtFs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=HGDam/mf46YVaaePusGnND4wrPyxKhB5w0wXtyFvsc2B0fmkgqioiIKnaDtaGGfgxD wOgo8Ts3DOVjCt792gYAJ8YIZJekQaoCGYiwS/gaRXSWRi3YL6R2ceIKg7qS5EDZg+XC PrY9zY2+zKuNT7YPezyeR9j+MFyr9rxAiUkl4= MIME-Version: 1.0 Received: by 10.227.69.134 with SMTP id z6mr1035486wbi.201.1282306200144; Fri, 20 Aug 2010 05:10:00 -0700 (PDT) Sender: julien.laffaye@gmail.com Received: by 10.216.13.133 with HTTP; Fri, 20 Aug 2010 05:09:59 -0700 (PDT) In-Reply-To: References: <20100819143830.GJ35140@azathoth.lan> <4C6DA0FA.7080203@DataIX.net> Date: Fri, 20 Aug 2010 14:09:59 +0200 X-Google-Sender-Auth: Q6g96JdPWIEV225YVxnXHubn-zk Message-ID: From: Julien Laffaye To: Ivan Voras Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: bapt@freebsd.org, Florent Thoumie , David Forsythe , Garrett Cooper , Tim Kientzle , freebsd-ports@freebsd.org Subject: Re: what next for the pkg_install rewrite X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Aug 2010 12:10:03 -0000 On Fri, Aug 20, 2010 at 12:00 PM, Ivan Voras wrote: > > And... both ideas are completely wrong. SQLite can be imported as a > single C file + header, which you must agree is practically the > optimum, and its license is "public domain" which is, if anything, > "freer" than BSDL and eminently compatible with it. > > And if we don't want to build a sqlite.so which can conflict with the version from ports, we can statically link libpkg to it. Incognito. > BDB in base offer exactly one (1) single "feature", if you can call it > that: storing and retrieving key-value binary blobs. It has no > practical concurrency support, it's locking is laughable and upgrading > data stored as binary blobs is even more nightmarish than maintaining > the current plist format (and it will lead to similar uglyness soon - > rather than upgrading the data piece called "x", I'm sure developers > will introduce new keys called "x_extdata", "x_moredata" etc etc). > If we are going key-value storage, I think that TinyCDB is worth a look. > > SQLite on the other hand solves all these in the following way: > > 1) stores proper records, not blobs > 2) has proper locking & concurrency support, ACID > 3) the database schema can be modified on the fly for upgrading - > fields can be added to existing table while preserving data. > 4) is endian-agnostic, 32/64-bit agnostic and portable (I mean the > database file) to an extremely large number of platforms. It is > already used as a system database in OS X, iPhone and Android. > > Note that I'm promoting SQLite to replace the /var/db/pkg/* tree of > directories and files with ad-hoc structure - all data in there should > be in one SQLite database, which is stored in a single file. > Indeed, all this points can help. The major drawback of SQLite, in my opinion, is the inclusion of long SQL statement in c code. So if we are going this way, we must have strict rules to avoid "polluting" the source. > [...] > > As for backward compatibility: basically it can be done in two ways: > 1) build a one-time converter that will read the present plist > database and output a new database or 2) build a compatibility layer > which would support both the old and the new format at the same time, > so people upgrading will continue to use their old data. I consider > the latter wasteful in terms of developer resources and because > bit-rot will eventually make support for the old format decrepit. > Agreed, a one time converter seems to be the solution. Especially if the transition occur between two major FreeBSD version (say, 9 - > 10). > > > 2. XML is a bad idea. Great in theory, wonderful in my browser, but a > > bloated plaintext file with a lot of complexity. I would prefer a > > database solution that could be dumped to a simple text file format if > > needed. > > XML, at least in my proposal, would not be used for the system package > database, but would replace (either in part or all with a single file) > today's "+" files in the packages, together with other purposes where > some metadata transfer is required. > > That said, I would also be happy with other self-described formats > like JSON. XML is the front runner because it's more standard > (industry standard, whether someone likes this fact or not!) and there > is already a parser in base. > The pro of XML here is that we have a parser in base, The con is its verbosity. But anyway the manifest is not meant to be human readable. Nevertheless, I prefer JSON, because it's less complex than XML and we can have a parser/emiter in a single 500SLOC .c file. And it looks sexier than XML, eh : http://pastebin.com/8hzPSSJC