Date: Thu, 29 Oct 2009 00:08:33 -0500 (CDT) From: Scott Bennett <bennett@cs.niu.edu> To: bf1783@googlemail.com, freebsd-questions@freebsd.org Subject: Re: lang/gcc43 and lang/gcc44 installation procedures broken after updates Message-ID: <200910290508.n9T58XqA014030@mp.cs.niu.edu>
next in thread | raw e-mail | index | archive | help
On Wed, 28 Oct 2009 09:19:08 +0000 "b. f." <bf1783@googlemail.com>
wrote:
>On 10/28/09, Scott Bennett <bennett@cs.niu.edu> wrote:
>> On Tue, 27 Oct 2009 11:28:51 +0000 "b. f." <bf1783@googlemail.com>
>> wrote:
>>>Scott Bennet wrote:
>...
>>
>> With one exception, I do not alter the
>> contents of the ports tree manually.
>...
>> I have not made alterations to any ports in the ports tree by hand. Any
>> changes that may have occurred would have to have happened during runs of
>> portmaster, portupgrade, or make(1) (as in "make deinstall && make
>> reinstall"
>...
>> I resorted to a "portsnap fetch extract" in case something
>> in my ports tree *had* gotten screwed up somehow.
>
>Right, I wasn't suggesting it was necessarily due to local changes to
>the Ports tree, although on the face of it that was possible, but that
>it may also have failed because, once in a while, binaries and other
>files belonging to the base system or ports get corrupted, and
>malfunction. This is usually due to hardware problems, user error,
>and occasionally, an OS or third-party software bug. The lang/gcc4?
>ports are lengthy and demanding builds, and are among the most likely
>to fail if such problems exist.
Yes, they are heavy-duty constructions. perl also tends to be huge
and complex.
>
>...
>> The only change I made was indicated by a comment that showed where
>> a lot of lines were deleted. If you really want all that junk, which
>> contained no error messages, I do still have it and can send it to you.
>> Nothing was rearranged into a different order, however.
>
>You may want to save it, so that it will be available if anyone
>decides to try to track down the problem.
>
I'll hang onto it for a while, but will eventually get rid of it
if no one wants it before I get around to deleting it in a few weeks
or so.
>>
>> I do not have MAKEFLAGS set when running portmaster or portupgrade.
>> If a particular port decides internally to run a parallel make, it appears
>> to do it as -j2. It appears that the lang/gcc?? ports work this way, too.
>>
>
>If parallel builds are not disabled in a port Makefile, or by you, and
>you have a multiple-cpu or multiple-core machine, then
>ports/Mk/bsd.port.mk uses:
>
>
># Multiple make jobs support
>.if defined(DISABLE_MAKE_JOBS) || defined(MAKE_JOBS_UNSAFE)
>_MAKE_JOBS= #
>.else
>.if defined(MAKE_JOBS_SAFE) || defined(FORCE_MAKE_JOBS)
>MAKE_JOBS_NUMBER?= `${SYSCTL} -n kern.smp.cpus`
>_MAKE_JOBS= -j${MAKE_JOBS_NUMBER}
I figured it must do something of the sort. The CPU is an old 3.4 GHz
P4 Prescott, so it has two logical processors, so MAKE_JOBS_NUMBER gets set
to 2. Given the handbook recommendations and my own observations, it seems
to me that the above method should actually multiply the value of
kern.smp.cpus by at least 2.5 for best performance. For CPUs on separate
cores, 3 is the recommended multiplier, but where HTT logical CPUs are
involved a multiplier somewhat lower than that is in order. On the Prescott
chips, 2.5 seems to work very well, so when I set MAKEFLAGS myself, I set
it to 5, which is 2.5 * kern.smp.scpus.
>.if defined(FORCE_MAKE_JOBS)
>BUILD_FAIL_MESSAGE+= "You have chosen to use multiple make jobs
>(parallelization) for all ports. This port was not tested for this
>setting. Please remove FORCE_MAKE_JOBS and retry the build before
>reporting the failure to the maintainer."
>.endif
>.endif
>.endif
>
>to do a parallel build. Since this feature is relatively new, and
>people are occasionally finding that it breaks port builds, then it is
>an obvious thing to try disabling in a case like this, where you have
>a demanding build, and some evidence that things are being done out of
>the proper order. In the future, you can disable this feature for a
>build by setting DISABLE_MAKE_JOBS=yes on the make command line, in
>the build environment, or in /etc/make.conf, e.g.:
>
>.if${.CURDIR:M*/usr/ports/lang/gcc44*}
>DISABLE_MAKE_JOBS=yes
>.endif
>
>>
>> portmaster long since created a backup package and deinstalled the
>> ports in question.
>
>Ok. I don't use portmaster often, but portupgrade will often restore
>an old installation of the port from the backup package automatically
>after a failure.
>
>...
>
>> I cannot begin to imagine why
>> it worked this way, but refused to work under portmaster or portupgrade.
>
>Occasionally a port exposes a bug in portmaster or portupgrade. This
>may be such a case, especially since Doug Barton made some recent
>changes to portmaster. But the most common reason for failure is that
>many ports, to enable easy maintenance, use sloppy flags like
>LDFLAGS=-L${LOCALBASE}/lib or CPPFLAGS=-I${LOCALBASE}/include, that
>may lead them to link against the older, already installed versions of
>themselves, or to include old versions of their own headers if they
>are present in the system. So it's always safer to deinstall a port
>_before_ attempting to build it, or to build the port in a clean
>sandbox as is done on many package-building clusters. portmaster and
>portupgrade choose not to do this, in order to shorten the process of
>recovering from a failed build, and to minimize the time during which
>a piece of software is unavailable to users, and this can lead to
>problems. I don't say that this happened in this case, but it is a
>possibility.
>
>> I guess I will just have to add "-x gcc\*" to the
>> "portmaster -x perl\*5.8.9\* -a" runs from now on, which is now possible
>> thanks to Doug Barton's portmaster enhancement that allows multiple "-x"
>> arguments, and do lang/gcc* updates by the old-fashioned method that worked
>> in this case. I'm not sure what to do if a situation arises like this for
>> a port that has many dependencies that would typically be better managed by
>> portmaster or portupgrade, however.
>
>You don't have to do it on the command line -- you can add the port to
>HOLD_PKGS in pkgtools.conf with portupgrade, or use a
I haven't been using portupgrade much lately. portmaster seems to
be the recommended tool, and it's certainly a lot faster than portupgrade,
which uses ruby18, another port broken by a recent update. (Hmm...that
reminds me...I should try reinstalling ruby18 without using portmaster or
portupgrade.) I do use portinstall to install packages because portmaster
cannot yet install packages.
>/var/db/pkg/*/+IGNOREME as described in portmaster(8). It's a bit of
Yes, but that method doesn't work for perl, and IIRC, it doesn't
work for lang/gcc?? either. The -x method does, however.
>a pain to manage large updates -- you may be better off fashioning
>your own tool. You could, for example, decide on a list of ports to
>update, and then determine the ports that depend upon them, and sort
>the lot into dependency order via:
>
>pkgdb -L (to be sure the pkgdb has got the dependency graph right)
>pkg_glob -r <insert list of ports to update here> | pkg_sort > /tmp/updatelist
>
>Now you could just feed this sorted update list to your own updating
>script, or use one of the existing tools. For example, if you wanted
>to use portupgrade, assuming that lang/gcc44 is midway down the update
>list, and you've got lang/gcc44 in HOLD_PKGS in pkgtools.conf:
>
>sed -n '1,/gcc-4.4/p' /tmp/updatelist | xargs portupgrade --batch -cfuv
>
>cd /tmp && pkg_create -b gcc-4.4* && make -C /usr/ports/lang/gcc44
>deinstall clean install && make -C /usr/ports/lang/gcc44 clean
>
>sed -n '/gcc-4.4/,$p' /tmp/updatelist | xargs portupgrade --batch -cfuv
>
>and youi've done the update in only a few steps. If you place it in
>a script, it will be even easier. With portmaster, using -a, -f, or
>-r, you could try interrupting or pausing the build at lang/gcc44,
>rebuilding that port by hand, and then resuming the build, with -R and
>the other appropriate flags if necessary.
>
Thanks much for the examples. I'll save your message for future
reference in case I need to resort to them or to modified versions of
them at some point.
Scott Bennett, Comm. ASMELG, CFIAG
**********************************************************************
* Internet: bennett at cs.niu.edu *
*--------------------------------------------------------------------*
* "A well regulated and disciplined militia, is at all times a good *
* objection to the introduction of that bane of all free governments *
* -- a standing army." *
* -- Gov. John Hancock, New York Journal, 28 January 1790 *
**********************************************************************
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200910290508.n9T58XqA014030>