From owner-freebsd-ports@FreeBSD.ORG Thu Jan 26 19:19:49 2012 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8EE321065670 for ; Thu, 26 Jan 2012 19:19:49 +0000 (UTC) (envelope-from m.seaman@infracaninophile.co.uk) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) by mx1.freebsd.org (Postfix) with ESMTP id C5C478FC13 for ; Thu, 26 Jan 2012 19:19:48 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [IPv6:2001:8b0:151:1:fa1e:dfff:feda:c0bb]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.5/8.14.5) with ESMTP id q0QJJhB7073521 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 26 Jan 2012 19:19:44 GMT (envelope-from m.seaman@infracaninophile.co.uk) X-DKIM: OpenDKIM Filter v2.4.3 smtp.infracaninophile.co.uk q0QJJhB7073521 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=infracaninophile.co.uk; s=201001-infracaninophile; t=1327605584; bh=IG1Qip/nz3xr/4tbnHrJLGukcPKbglZ+Zyc8w0yhj/E=; h=Message-ID:Date:From:MIME-Version:To:Subject:References: In-Reply-To:Content-Type:Cc; b=LQIF2zcqLwuNS/Wzqg2TA0T6yEJziTF6Fgngkp6blMYLLVylfbAXLf0HH+ZQjdTQX 8lsTRuax1Ht2LE5XPcMsRLuDXpUEUZKrofHtbpgnJv6PJwZrtu8rYxVxFBidF/k5I3 5cnVPXFO81qz/CpMjCE+UbK8qKVcHZ+sdgYuEW+0= Message-ID: <4F21A748.6010307@infracaninophile.co.uk> Date: Thu, 26 Jan 2012 19:19:36 +0000 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0) Gecko/20111222 Thunderbird/9.0.1 MIME-Version: 1.0 To: freebsd-ports@freebsd.org References: <4F177264.3090708@freebsd.org> <4F17DB1C.6080503@infracaninophile.co.uk> <4F193FD5.8070208@infracaninophile.co.uk> <20120121204614.GH4729@lonesome.com> <4F1BE7B4.6090702@infracaninophile.co.uk> In-Reply-To: <4F1BE7B4.6090702@infracaninophile.co.uk> X-Enigmail-Version: 1.3.4 OpenPGP: id=60AE908C Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig9CDC4A00F9A907A2F8254622" X-Virus-Scanned: clamav-milter 0.97.3 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.7 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk Subject: Re: NOT_FOR_ARCHS considered harmful [was: with the cvs history? trying to help INDEX builds.] X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jan 2012 19:19:49 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig9CDC4A00F9A907A2F8254622 Content-Type: multipart/mixed; boundary="------------030703060201030600050905" This is a multi-part message in MIME format. --------------030703060201030600050905 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 22/01/2012 10:40, Matthew Seaman wrote: > On 21/01/2012 20:46, Mark Linimon wrote: >> tl;dr: I want to switch the default assumption we're making. >> >> IMHO when new ports come into the tree, we should make our default >> assumption that we will try to build them on amd64 and i386. For case= s >> that this does not hold, we consider this Bad and committer-must-fix. >> For the tier-2s, we shift the default assumption to "only set it to >> buildable once it has been shown to be so". So, the burden of proof >> shifts the other way: to a user of a tier-2 to claim "I tried this and= >> it works", rather than portmgr saying "we tried this and it doesn't wo= rk". >=20 > Doesn't your proposed change in semantics of the 'FOR_ARCHS' stuff mean= > that over time, as other architectures become more popular, most ports > will have to have an explicit 'ONLY_FOR_ARCHS' setting? If the default= > effectively becomes 'ONLY_FOR_ARCHS=3D i386 amd64' then as ports are sh= own > to work on different platforms they will need an ONLY_FOR_ARCHS line in= > their Makefiles listing where they are known to work? Or else the port= s > becomes effectively i386 / amd64 only? >=20 >> (Of course, for things like p5-* it doesn't really matter; if perl >> builds, to a first approximation they'll build as well. I'm talking >> about the things like biology/, deskutils/, games/, math/, science, >> x11*/, and so forth.) >> >> What do people think? >=20 > There are a lot of ports where the distinction between CPU architecture= s > is pretty much irrelevant. I can't see portmaster(8) (for example) > failing to work anywhere the base system works. >=20 > I was thinking about this a while back. Test the contents of packages > to see if they install any object code -- ports/129210 -- and mark the > ones that don't as arch-independent in some way (CATEGORIES+=3D arch-in= dep > perhaps?) So, apropos Mark's comments earlier, and after a few false starts and a lot of head-scratching here is a patch to address many of his points. It does basically three things: * Removes the NOT_FOR_ARCHS variable and associate functionality. * Creates a new BROKEN_OSARCHS variable. This is largely syntactic sugar, meaning that it is no longer necessary to use .include in order to test ${ARCH} or ${OSVERSION} * Creates a new SUPPORTED_OSARCHS variable. This allows maintainers to indicate which architectures they will support, and whether the package build cluster should routinely build packages for that port. A full patch removing NOT_FOR_ARCHS from the ports is at: http://www.infracaninophile.co.uk/not_for_archs.diff Comments? Critiques? Now for the more long-winded explanation bit.... NOT_FOR_ARCHS is frequently misused to indicate temporary build or installation failures on particular architectures. The _FOR_ARCHS stuff should be really only be used to indicate ports designed to work on specific CPU architectures. ONLY_FOR_ARCHS exists for this purpose. The problem with extending it to cover temporary failures is that the package build cluster will never retry building that port, so there's no way to tell if changes to the toolchain or updates to the port have subsequently fixed the problem. This is where the BROKEN_OSARCHS variable is useful. This is a list of achitecture-OS version pairs corresponding to the different environments used in the package build cluster. Thus you can easily mark a port as broken on sparc64-7, rather than having to use a construction like this: =2Einclude =2Eif ${ARCH} =3D=3D "sparc64" && ${OSVERSION} < 800000 BROKEN=3D compiler dumps core =2Eendif =2Einclude (Not using bsd.port.{pre,post}.mk so much should help a bit with index building times too.) It's also rather easier to grep for BROKEN_OSARCHS than it is for a construct like the above. As BROKEN_OSARCHS ultimately uses the pre-existing BROKEN variable, setting TRY_BROKEN will allow these ports to be retested. Finally there is SUPPORTED_OSARCHS, which can be used to indicate policy about where a port is known to usable. For regular users it doesn't do anything except print a warning message if an attempt is made to compile a port on an unsupported architecture/OS combination. For the ports build cluster, conditional on PACKAGE_BUILDING being set, it turns off building on unsupported environments by setting IGNORE. As the patch stands at the moment, every port will be using the default, which is to support these architecture/OS combinations: i386-7 i386-8 i386-9 amd64-7 amd64-8 amd64-9 but the defaults can be modified easily or port-specific settings can be added. There is also a TRY_UNSUPPORTED knob to turn on building packages on all architectures / OS versions. Finally, all this testing the CPU architecture and the OS version makes no sense at all for ports that don't install any compiled binaries, such as shell scripts. Those ports can define a variable ARCH_INDEP to opt out of the SUPPORTED_OSARCHS thing altogether. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate JID: matthew@infracaninophile.co.uk Kent, CT11 9PW --------------030703060201030600050905 Content-Type: text/plain; x-mac-type="0"; x-mac-creator="0"; name="bsd.port.mk.diff" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="bsd.port.mk.diff" Index: bsd.port.mk =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /home/ncvs/ports/Mk/bsd.port.mk,v retrieving revision 1.703 diff -u -u -r1.703 bsd.port.mk --- bsd.port.mk 22 Jan 2012 15:41:33 -0000 1.703 +++ bsd.port.mk 26 Jan 2012 18:20:31 -0000 @@ -207,17 +207,69 @@ # # Set these if your port only makes sense to certain architectures. # They are lists containing names for them (e.g., "alpha i386"). -# (Defaults: not set.) +# (Defaults: unset) # # ONLY_FOR_ARCHS # - Only build ports if ${ARCH} matches one of these. -# NOT_FOR_ARCHS - Only build ports if ${ARCH} doesn't match one of these= =2E # ONLY_FOR_ARCHS_REASON # ONLY_FOR_ARCHS_REASON_${ARCH} # - Reason why it's only for ${ONLY_FOR_ARCHS}s -# NOT_FOR_ARCHS_REASON -# NOT_FOR_ARCHS_REASON_${ARCH} -# - Reason why it's not for ${NOT_FOR_ARCHS}s +# +# Show architecture and OS major version combinations where ported +# software is expected to operate correctly and for which packages +# should be built. Port maintainers are expected to address problems +# with ports on supported OS versions and architectures. Where +# physically feasible, all Tier-1 architectures and all OS versions +# under active support by the FreeBSD project must be supported. +# Architecture and OS combinations not listed as supported may or may +# not work, and maintainers need not feel obliged to address reported +# problems. +# +# OS version and architecture combinations are expressed as +# ${ARCH}-${OSMAJOR} (eg. amd64-9, i386-10). The wildcard 'ALL' may +# be used to indicate support for the default set of either +# component. Thus i386-ALL means supported on all active supported +# major OS versions for i386 architecture, whereas ALL-9 means +# supported on all Tier-1 architectures but only for FreeBSD 9.x. The +# default setting is ALL-ALL +# +# Ports that install no architecture dependent files (eg. shell +# scripts) can define ARCH_INDEP to opt-out of all supported OS +# version and architecure tests, implicitly declaring that they are +# supported on all platforms. +# +# ARCH_INDEP - Define this to declare that this port installs identic= al +# content irrespective of OS version and system architec= ture +# SUPPORTED_OSARCHS +# - List of supported architecture-OS combinations this +# port is supported for. Only build packages if +# ${ARCH}-${OSMAJOR} matches one of these. (Default: +# ALL-ALL) +# SUPPORTED_REASONS +# - List of reasons why architecture is or is not supporte= d +# on various platforms. The format is :"" +# The first match is selected, so order from most to lea= st +# specific. +# Example: +# ALL-7:"supported on standard architectures for FreeB= SD 7.x" +# ALL-8:"supported on standard architectures for FreeB= SD 8.x" +# ALL-ALL:"part of base system" +# (Default: unset) +# +# Set these if your port temporarily does not work on certain +# architectures and OS combinations due to programming errors or other +# unintentional problems. Values are lists of architecture-OS +# combinations as above. (Defaults: unset) +#=20 +# BROKEN_OSARCHS +# - List of architecture-OS combinatins where port is +# known not to work. See BROKEN above. +# BROKEN_REASONS +# - List of reasons why port is broken on +# ${ARCH}-${OSMAJOR}. The format is the same as for +# SUPPORTED_REASONS above. +# (Default: unset) +# # IA32_BINARY_PORT # - Set this instead of ONLY_FOR_ARCHS if the given port # fetches and installs compiled i386 binaries. @@ -1195,6 +1247,11 @@ .endif .endif =20 +# Get the operating system major version +.if !defined(OSMAJOR) +OSMAJOR=3D ${OSVERSION:C,.....$,,} +.endif + MASTERDIR?=3D ${.CURDIR} =20 .if ${MASTERDIR} !=3D ${.CURDIR} @@ -3126,21 +3183,8 @@ __ARCH_OK?=3D 1 .endif =20 -.if defined(NOT_FOR_ARCHS) -.for __NARCH in ${NOT_FOR_ARCHS} -.if ${ARCH:M${__NARCH}} !=3D "" -.undef __ARCH_OK -.endif -.endfor -.endif - .if !defined(__ARCH_OK) -.if defined(ONLY_FOR_ARCHS) -IGNORE=3D is only for ${ONLY_FOR_ARCHS}, -.else # defined(NOT_FOR_ARCHS) -IGNORE=3D does not run on ${NOT_FOR_ARCHS}, -.endif -IGNORE+=3D while you are running ${ARCH} +IGNORE=3D is only for ${ONLY_FOR_ARCHS}, while you are running ${ARCH} =20 .if defined(ONLY_FOR_ARCHS_REASON_${ARCH}) IGNORE+=3D (reason: ${ONLY_FOR_ARCHS_REASON_${ARCH}}) @@ -3148,12 +3192,143 @@ IGNORE+=3D (reason: ${ONLY_FOR_ARCHS_REASON}) .endif =20 -.if defined(NOT_FOR_ARCHS_REASON_${ARCH}) -IGNORE+=3D (reason: ${NOT_FOR_ARCHS_REASON_${ARCH}}) -.elif defined(NOT_FOR_ARCHS_REASON) -IGNORE+=3D (reason: ${NOT_FOR_ARCHS_REASON}) .endif =20 +# Check the machine architectures. Assume ports are only supported on +# Tier-1 architectures unless maintainers commit to supporting other +# achitectures and they have been tested and shown to work there. Skip a= ll +# this for ports that install entirely architecture independent files. + +.if !defined(ARCH_INDEP) +SUPPORTED_OSARCHS?=3D ALL-ALL +SUPPORTED_REASONS?=3D ALL-ALL:"standard Tier-1 architecture and supporte= d OS" + +SUPPORTED_OSARCHS_DEFAULT=3D amd64-7 amd64-8 amd64-9 \ + i386-7 i386-8 i386-9 + +_ARCH_OS=3D ${ARCH}-${OSMAJOR} +_ARCH_WILD=3D ALL-${OSMAJOR} +_WILD_OS=3D ${ARCH}-ALL +_WILD_WILD=3D ALL-ALL + +# Id the current environment one that would be supported by default? +. for _osarch_def in ${SUPPORTED_OSARCHS_DEFAULT} +. if ${_ARCH_OS} =3D=3D ${_osarch_def} +__DEFAULT_OSARCH=3D 1 +. endif +. endfor + +. for _osarch in ${SUPPORTED_OSARCHS} +. if !defined(__SUPPORTED) +. if ${_ARCH_OS} =3D=3D ${_osarch} +__SUPPORTED=3D ${_ARCH_OS} +. elif defined(__DEFAULT_OSARCH) +. if ${_WILD_WILD} =3D=3D ${_osarch}=20 +__SUPPORTED=3D ${_WILD_WILD} +. elif ${_ARCH_WILD} =3D=3D ${_osarch} +__SUPPORTED=3D ${_ARCH_WILD} +. elif ${_WILD_OS} =3D=3D ${_osarch} +__SUPPORTED=3D ${_WILD_OS} +. endif +. endif +. endif +. endfor + +. for _supported_reason in ${SUPPORTED_REASONS} +. if !defined(__SUPPORTED_REASON) +_osarch=3D ${_supported_reason:C,:.*$,,} +_reason=3D ${_supported_reason:C,^[^:]+:",,:C,"$,,} + +. if ${_ARCH_OS} =3D=3D ${_osarch} +__SUPPORTED_REASON=3D ${_reason} +. elif defined(__DEFAULT_OSARCH) +. if ${_WILD_WILD} =3D=3D ${_osarch}=20 +__SUPPORTED_REASON=3D ${_reason} +. elif ${_ARCH_WILD} =3D=3D ${_osarch} +__SUPPORTED_REASON=3D ${_reason} +. elif ${_WILD_OS} =3D=3D ${_osarch} +__SUPPORTED_REASON=3D ${_reason} +. endif +. endif +. endif +. endfor + +# Don't build packages for unsupported architectures +. if defined(PACKAGE_BUILDING) && !defined(TRY_UNSUPPORTED) +. if !defined(__SUPPORTED) +IGNORE=3D unsupported: ${SUPPORTED_OSARCHS} does not match ${_ARCH_OS} +. if defined(__SUPPORTED_REASON) +IGNORE+=3D (reason: ${__SUPPORTED_REASON}) +. endif +. endif +. endif +.endif # !defined(ARCH_INDEP) + + +.if defined(BROKEN_OSARCHS) + +. for _osarch in ${BROKEN_OSARCHS} +. if !defined(__BROKEN) +. if ${_ARCH_OS} =3D=3D ${_osarch} +__BROKEN=3D ${_ARCH_OS} +. elif defined(__DEFAULT_OSARCH) +. if ${_WILD_WILD} =3D=3D ${_osarch}=20 +__BROKEN=3D ${_WILD_WILD} +. elif ${_ARCH_WILD} =3D=3D ${_osarch} +__BROKEN=3D ${_ARCH_WILD} +. elif ${_WILD_OS} =3D=3D ${_osarch} +__BROKEN=3D ${_WILD_OS} +. endif +. endif +. endif +. endfor + +. for _broken_reason in ${BROKEN_REASONS} +. if !defined(__BROKEN_REASON) +_osarch=3D ${_broken_reason:C,:.*$,,} +_reason=3D ${_broken_reason:C,^[^:]+:",,:C,"$,,} + +. if ${_ARCH_OS} =3D=3D ${_osarch} +__BROKEN_REASON=3D ${_reason} +. elif defined(__DEFAULT_OSARCH) +. if ${_WILD_WILD} =3D=3D ${_osarch}=20 +__BROKEN_REASON=3D ${_reason} +. elif ${_ARCH_WILD} =3D=3D ${_osarch} +__BROKEN_REASON=3D ${_reason} +. elif ${_WILD_OS} =3D=3D ${_osarch} +__BROKEN_REASON=3D ${_reason} +. endif +. endif +. endif +. endfor + +. if defined(__BROKEN) +BROKEN=3D on ${BROKEN_OSARCHS}; ${__BROKEN} matches ${_ARCH_OS} +. if defined(__BROKEN_REASON) +BROKEN+=3D (reason: ${__BROKEN_REASON}) +. endif +. endif + +.endif # defined(BROKEN_OSARCHS) + +.if !target(check-support) +check-support: +. if defined(ARCH_INDEP) + @${ECHO_MSG} "=3D=3D=3D> Supported anywhere: architecture-independent = content" +. else +. if defined(__SUPPORTED) +. if defined(__SUPPORTED_REASON) + @${ECHO_MSG} "=3D=3D=3D> Supported on ${_ARCH_OS}: ${__SUPPORTED_REASO= N}" +. endif +. else + @${ECHO_MSG} "=3D=3D=3D> WARNING: this port is not supported on ${_ARC= H_OS}" +. if defined(__SUPPORTED_REASON) + @${ECHO_MSG} "=3D=3D=3D> (reason: ${__SUPPORTED_REASON})" +. else + @${ECHO_MSG} "=3D=3D=3D> (reason: not properly tested and may or may n= ot work)" +. endif +. endif +. endif .endif =20 # Check the user interaction and legal issues @@ -4266,8 +4441,8 @@ _SANITY_SEQ=3D ${_CHROOT_SEQ} pre-everything check-makefile \ check-categories check-makevars check-desktop-entries \ check-depends identify-install-conflicts check-deprecated \ - check-vulnerable check-license buildanyway-message \ - options-message + check-vulnerable check-license check-support \ + buildanyway-message options-message _FETCH_DEP=3D check-sanity _FETCH_SEQ=3D fetch-depends pre-fetch pre-fetch-script \ do-fetch post-fetch post-fetch-script --------------030703060201030600050905-- --------------enig9CDC4A00F9A907A2F8254622 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk8hp08ACgkQ8Mjk52CukIwvPQCdFjx6F6pbSzm7SJogw3h3Ra1L ZxkAn2PspxQh0YPoKRNrmPUgFW3Ds4CH =ThUJ -----END PGP SIGNATURE----- --------------enig9CDC4A00F9A907A2F8254622--