Date: Fri, 20 Aug 2010 00:25:49 -0400 From: jhell <jhell@DataIX.net> To: Garrett Cooper <yaneurabeya@gmail.com> Cc: bapt@freebsd.org, Florent Thoumie <flz@freebsd.org>, Julien Laffaye <jlaffaye@freebsd.org>, David Forsythe <dforsyth@freebsd.org>, Garrett Cooper <gcooper@freebsd.org>, Tim Kientzle <kientzle@freebsd.org>, Ivan Voras <ivoras@freebsd.org>, freebsd-ports@freebsd.org Subject: Re: what next for the pkg_install rewrite Message-ID: <4C6E03CD.6040003@DataIX.net> In-Reply-To: <A4FFBD88-8830-4BB3-8765-AC7BAB8AC91C@gmail.com> References: <20100819143830.GJ35140@azathoth.lan> <AANLkTimY=FJas-oXkWwO07QtaD%2BGrLockgJ_SZQJ7UHM@mail.gmail.com> <AANLkTimhh2vOtXUb-frzWcZmANWyEC7oPtTgepzvOtSB@mail.gmail.com> <4C6DA0FA.7080203@DataIX.net> <AANLkTi=h_GdAFbZ2X0agCAtyLLiwNrMoLx_ZunhBBx2=@mail.gmail.com> <AANLkTinn4utHttMPsCN1GKp1dTPOdTJAtLgG3m8BfLgw@mail.gmail.com> <4C6DE8DB.8010202@DataIX.net> <A4FFBD88-8830-4BB3-8765-AC7BAB8AC91C@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 08/19/2010 23:08, Garrett Cooper wrote:
> On Aug 19, 2010, at 7:30 PM, jhell wrote:
>
>> On 08/19/2010 21:26, Garrett Cooper wrote:
>>> On Thu, Aug 19, 2010 at 3:10 PM, Ivan Voras <ivoras@freebsd.org>
>>> wrote:
>>>>> On 19/08/2010, jhell <jhell@dataix.net> wrote:
>>>>>>> Adding to this I would like to see a central database
>>>>>>> created for packages that have been removed like in
>>>>>>> Slackware Linux. They keep a file in
>>>>>>> /var/log/preserved_packages with a flat text format with
>>>>>>> the file name looking like:
>>>>>>>
>>>>>>> ${PORTNAME}-${PORTVERSION}${PORTREVISION}-`date
>>>>>>> +%Y%m%d%H%M%S`
>>> Please no. We need another ad hoc text format like we need holes
>>> in our head.
>>>
>>
>> You may have misunderstood or maybe not the intention behind that
>> file.
>>
>> This is just simply a log file of the transactions that were
>> performed upon package deletion and nothing more but just a way for
>> the user to look back and say "HEY! how did that get there!." or
>> "where in the ``jhell'' did that file come from!" that they can
>> simply grep the package removal logs to find out.
>>
>> -- Shameless plug with my email name put in for humor.
>
> :)
>
>> It is also really handy if you remove some packages that somehow
>> the depends had been messed up and later on your having a problem
>> with a missing lib, easy enough to grep the removal logs to find
>> out what package held that file. Especially useful if you only use
>> binary packages.
>
> This is part of the request that someone was making for a feature
> like Gentoo's world file on the forums:
> http://forums.freebsd.org/showthread.php?t=16963 .
>
> Personally it's one of the takeaways I like about Gentoo's portage
> system because it's easy to track what I as a user installed
> manually, and hence, it's easy to track what can be removed (instead
> of using pkg_cutleaves) if we have a `emerge -C` (package dependency
> [dist]clean equivalent). It also makes it easier for admins to mass
> install packages on multiple machines using a smaller `distroot'
> install binary base, so all I would have to do is:
>
> 1. Install prebuilt version of FreeBSD with sysinstall / ad hoc
> installer method. 2. Say, pkg_add <install all, force options> 3. Do
> whatever I need to do to configure the machine.
>
> Done.
>
> That would make system administration really easy and slick, and
> would improve the setup process for corporations that build on a
> static FreeBSD base for several releases and have varying packages
> for several bits. I know if my group did it, things wouldn't be such
> a mess..
>
> By the way... /var/lib/portage/world is a simple text file composed
> line by line like:
>
> <pkg-origin-a> <pkg-origin-b>
>
> i.e.
>
> devel/gcc46 lang/python26 www/firefox36
>
> etc. Simple and easy to understand, and easy to modify :).
>
>> There is a lot of information that can be logged, especially with
>> '-v', I personally do not think we or anyone for that matter should
>> pass up that opportunity to make sure the information is collected
>> rather than leaving it up to the user to redirect or script(1) the
>> output every time which they would still or should be able to do.
>>
>> Another approach that I have not seen talked about here is a
>> proposed directory layout. I think before 'unless I missed it' that
>> someone jumps into this, some standards & goals should be made and
>> made quite loudly as to attract as much public opinion and
>> suggestions as possible for what works, what does not & what people
>> would like to see.
>>
>> Something of this magnitude like changing packaging databases and
>> directory structures and all that involved needs a central place
>> and a clear, clean plan to be developed properly. I personally do
>> not see this list anymore as a proper place to discuss it.
>> packaging@ list request ? so this can all be centralized.
>
> I agree that it's high time that a freebsd-packaging@ list be
> created. sysinstall has its own list now -- we should have one for
> the packaging software too :).
>
>> PS: I have been toying with the idea of the directory layout just
>> for spurring thoughts of others. http://bit.ly/aNLhNU but until
>> there is a central place for these things it does not mean much.
>
>
> I think that you're adding unnecessary complexity to the overall
> issue. It really doesn't make sense for me to install packages that
> aren't available for my architecture for one (in particular today),
> unless you were thinking of serving up this data on an NFS server,
> but even then it doesn't make sense because almost all of these files
> are hardcoded to exist at ${LOCALBASE} when built as ports, so
> setting it to another location would be problematic. Other things
> would need to be done before you could get to this stage.
Thinking of not only a bootp server here that serves up or can serve up
a mass amount of different machines it would make it possible for them
to have local-amd64, local-i386 served off to specific machines with
specific package databases. But then again that type of complexity is
probably not needed as a whole but is pretty keen.
Also I was building upon the idea of getting the packages directly out
of the package database root so other information can be collected there
like XML/XSL log files or SQLite databases without a packaging tool
having to explicitly worry about excluding those from being packages.
Merely just serving as a open workspace for other additions. I am
certain there is plenty of other ways to go about this very same thing.
In a case of moving the contents for say from i386 to amd64 you could
blindly copy your whole system over and upgrade it in place while
watching directories disappear from one arch directory and appear in the
correct one for the new machine. But again complex and really how often
does this scenario happen.... edge case, but to say the least still
pretty keen.
>
> Also, many of the ports installed are prefixed with the package name,
> which is different for multiple ports. Example:
>
> $ ls /var/db/pkg/python* python-2.6,2/ python26-2.6.5_1/
> python31-3.1.2_1/
>
This has always bothered me. I strongly believe in not renaming a
package name for the sake of it being just a different version but with
the understanding that in the case of ports we are left with very little
room for naming at this point in order to make binary packaging possible
on a larger scale of ports.
This is sort of why just for package database sake I was thinking of
/var/db/pkg/{packagename}/version/contents as a structure. But this does
not at all take care of building multiple versions of binary packages
from ports which is why I always come back to the idea of a package
containing the arch that its being built for and the binary package
distribution tree being revamped to take that into account along with
ports being reorganized to do the same thing.
ports/packages/{i368,amd64,ppc,(...)}/...
ports/packages/noarch/... Relieves the need for multiples
> Having a concept of multiple versions in ports would require a major
> overhaul to get things to work in a Gentoo like method, and I'm not
> sure how many people would be particularly keen in doing this
> (especially when there are name collisions in installed package
> files).
>
> Oh, and there was the tunable packages idea (what I originally called
> `fat packages', but Julien has done work on `fat packages', so I'll
> yield to that usage :P), where the contents of packages could be
> tuned according to config options requested by end-users and/or set
> by package maintainers, to get the granularity of distributing
> orthogonal features for packages, like xorg-server-hal-support
> xorg-server-no-hal-support (or whatever the proper title would be).
I have not seen personally other OS's that are trying to package
software by use of names of options that they are compiled with. This
IMO is too complex to achieve and makes me believe that we have lost
track of the minimal that needs to be accomplished for the end user to
have a bare minimum working generic install from point of distribution.
This is also a lot to ask of FreeBSD.org to host all these specialized
compilations of different software too.
Something that might ease this would be the ability to build from ports
without installing, and creating a package in-place so as to ease
multiple package builds with dynamic creation of the end resulting
package name. And when xorg-server-no-hal-support-321.tbz gets installed
? it should register as xorg-server-321 but be able to serve up its
build options to the user with the use of something like pkg_info.
Id like to copyright and trademark this idea, not fully thought through
but Ill call it the (c)PIT(tm) ME! & the FreeBSD project of course. As
in Package Install Targets. Simply a file parsed by an installer like
pkg_install but recognized by its extension ".pit". Containing specific
actions to be performed and names of packages to be installed and where
those packages are located either direct paths on a system or full URLs
to where the package can be fetched. This (c)PIT file could be generated
and then packaged within an archive of packages and distributed as a
whole. Since the (c)PIT file can contain external URLs to fetch packages
from then this would relieve the worry about distributing proprietary
licensed binary packages as well. The file could also hold the options
that each package it references to.
Purely Theoretical at this point but up for adoption.
>
> The emphasis that Florent made too was to remove crud in pkg_install
> and libpkg and get things down to more of a library form so we could
> develop thin wrappers above pkg_install with logical functions (like
> apt-get, yum, etc does with fetching, whereas rpm does with
> installation, etc), instead of maintaining pkg_install the way it is
> today.
I like this idea a lot!. Centralized place as to where anyone and
anything can call subroutines or other hooks! priceless! Not only would
it be easier to work with but scalable beyond words (given if its
designed right).
>
> Thanks, -Garrett
No no, Thank you, ;)
--
jhell,v
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4C6E03CD.6040003>