Date: Sun, 14 Jul 2013 12:25:13 +0100 From: Matthew Seaman <matthew@FreeBSD.org> To: Devin Teske <dteske@freebsd.org> Cc: Baptiste Daroussin <bapt@freebsd.org>, "Teske, Devin" <Devin.Teske@fisglobal.com>, Peter Wemm <peter@wemm.org>, "current@FreeBSD.org" <current@freebsd.org> Subject: Re: [HEADSUP] No more pkg_install on HEAD by default Message-ID: <51E28A99.5000904@FreeBSD.org> In-Reply-To: <13CA24D6AB415D428143D44749F57D7201FC547A@ltcfiswmsgmb21> References: <20130712231637.GS85556@ithaqua.etoilebsd.net> <13CA24D6AB415D428143D44749F57D7201FC2DBD@ltcfiswmsgmb21> <20130713080732.GV85556@ithaqua.etoilebsd.net> <13CA24D6AB415D428143D44749F57D7201FC3AA2@ltcfiswmsgmb21> <13CA24D6AB415D428143D44749F57D7201FC3C92@ltcfiswmsgmb21> <CA%2B7WWSe7C2UXAsEX3OZH7nPJJbxotpQ_iYBxgake4R68sbUtLA@mail.gmail.com> <13CA24D6AB415D428143D44749F57D7201FC3FAA@ltcfiswmsgmb21> <CAGE5yCoH2auer_kKpUT_caFUZPpVM5TdAFH5tJcGgF4Ji12f0g@mail.gmail.com> <13CA24D6AB415D428143D44749F57D7201FC4E0D@ltcfiswmsgmb21> <51E26FD3.4020208@FreeBSD.org> <13CA24D6AB415D428143D44749F57D7201FC547A@ltcfiswmsgmb21>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) ------enig2SGGEWPOMTGXFIDTLLKGF Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 14/07/2013 11:12, Teske, Devin wrote: > Interesting. I notice that (while looking ahead to see a prefix: of > /usr/local in the +MANIFEST), the tarball itself has files that include= > /usr/local in their path. Yes -- we consider the $PREFIX to be 'baked into' the package at compile time. You can relocate a package to some extent at installation time -- so for instance you can install into a jail or chroot from outside it -- but the software would expect all the files to be arranged as shown in the package at runtime. > Does pkg handle packages where the prefix (in +MANIFEST) is > "/usr/local" _but_ "tar tf" of the unxz'd tarball is _not_ "/usr/local"= ? > (e.g. pkg_install style) or is this the new way going forward? No -- it always wants a fully qualified path there. > So just to give you a better idea of how we manage packages here. >=20 > We've realized that if you want to package for FreeBSD (in 9 and > older), you could get by alright if you knew how to create a > +CONTENTS file from scratch. For RedHat, it's the SPECFILE. And now > for FreeBSD 10, it looks an awful lot like a SPECFILE (they are both > in-fact YAML). >=20 > So rather than teach the people here how to use tools, I taught them > what I think is more important... how to manage +CONTENTS, SPECFILE, > and now up-and-coming, +MANIFEST. >=20 > However, I'd be lying if I said I taught them *all* the gory details. > In reality, for +CONTENTS they edit a PLIST which is essentially > +CONTENTS without the "@comment MD5:" entries. For SPECFILE, they > manage the full thing except there's a small section that is the > standard RPM bootstrap that is labeled as "do not touch". >=20 > From what you posted of +COMPACT_MANIFEST was nice, except it lacks > the list of files. There's actually two lists: files and directories. The principle difference being that the files list contains SHA256 checksums of each file. (Although the files list does also contain sym-links etc, which don't have a checksum). There are other data in the full manifest but not in the compact one -- pre- and post- installation or deinstallation scripts are the most important bits not yet mentioned. > The basic principle here is that if you have to maintain a list of > files, and that list ends up being compiled into a +MANIFEST, why not > just teach your peers how to read/write/maintain +MANIFEST files (in > version control of course) so that there are never any mysteries as > to how the package performs. >=20 > I know this sounds screwy... but (as with our other YAML files), it's > really nice because (as is the case with SPECFILE), we > version-control things as-close to the end-product as possible (take > for example the case of "pre-install" or "post-install" et cetera > scripts -- I'm sure a tool like "pkg create" or "pkg_create" or other > could do a good job of assembling the pieces into the +MANIFEST, but > then it's not being version-controlled as closely. Actually, 'pkg create' basically takes the manifest and uses it to build the package. It has a special mode to combine data as presented during ports compilation into a manifest -- again, this is a compatibility thing. If the ports didn't have to support pkg_tools still we could generate package metadata more directly in the format needed by the manifest. > Before this workflow, mistakes were rampant and there wasn't much > hope of wrangling the "oops, forgot to touch the PLIST" or "oops, > forgot to update the post-install script" mistakes. >=20 > I see a nice bright future in re-working my "pkgbase" to be able to > supplant metadata into a revision-controlled +MANIFEST. >=20 > Ideally, I don't want them to have to be burdened with maintaining > both a +MANIFEST and +COMPACT_MANIFEST, so it seems most logical to > generate one from the other (the latter being the revision-controlled > copy sans-meta-data; the meta-data is added at "make" time before > then running validations and generating the tarball). Internally, the +COMPACT_MANIFEST is generated from the +MANIFEST -- probably best to think in terms of dividing the manifest into sections, some of which (lists of files, install scripts) your dev people would actively modify, some of which can be automatically generated by pkg(8) (shlib provides/requires tracking[*]) and the rest as static boilerplate you can just paste together as required to create a manifest. [*] Actually, this is automatically generated at the time the package is assembled into a tarball. You don't need to fill in that section of the manifest eplicitly. > The problem of breakage to the system by API changes is less > important, because we track security releng branches and use binaries > so changes are very slow to creep in. But since *can* use a different > framework for each/every major release branch, we can track API > changes quite easily. Ah. Remember that pkg(8) isn't part of the base system and is developed on a timetable independent of the timetable used by FreeBSD itself. When we release a new version of pkg, it would be applied simultaneously on all branches and supported releases. Of course, if you have your own private repository system you can control the version of pkg supplied by = it. > I'm aware that only slow moving dinosaurs like us are interested in > such backward-thinking mechanics... but this stuff really works for > us, and I'm certain that digesting +MANIFEST will be very beneficial > to our growth. One of our aims when developing pkg(8) has been to make it possible to have as much or as little interaction with the official FreeBSD package building clusters as you want. You could use official packages exclusively; or you could create a repo containing just the software packages crucial to you with local customizations, and install the rest from the official repos; or you can build everything yourself. Finally, it should be possible for companies or software projects to release just their own software in pkg(8) format, and for an administrator to select that for installation on their system rather than the default version (if any) from ports. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. PGP: http://www.infracaninophile.co.uk/pgpkey ------enig2SGGEWPOMTGXFIDTLLKGF Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlHiiqIACgkQ8Mjk52CukIynrACdHRBC50/4JNsl1eVdpzZcZi3b UaAAnjvKmaZ2bSKzxC6vtkZY4nqU0HVd =GSQd -----END PGP SIGNATURE----- ------enig2SGGEWPOMTGXFIDTLLKGF--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?51E28A99.5000904>