From owner-freebsd-current@FreeBSD.ORG Sun Sep 2 00:04:45 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B519106566B; Sun, 2 Sep 2012 00:04:45 +0000 (UTC) (envelope-from tim@kientzle.com) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id D5A148FC08; Sun, 2 Sep 2012 00:04:44 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id q8204ct0021294; Sun, 2 Sep 2012 00:04:38 GMT (envelope-from tim@kientzle.com) Received: from [192.168.2.143] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id ww3aftsqzkgnq6ehf74679pwzi; Sun, 02 Sep 2012 00:04:38 +0000 (UTC) (envelope-from tim@kientzle.com) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=iso-8859-1 From: Tim Kientzle In-Reply-To: <50426493.7050302@FreeBSD.org> Date: Sat, 1 Sep 2012 17:04:36 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <18B2DCFF-3769-46BF-9801-AD06E0A75A40@kientzle.com> References: <97612B57-1255-4BB3-A6D3-FC74324C6D67@FreeBSD.org> <503FF0EE.2020605@FreeBSD.org> <20120831095910.GQ64447@ithaqua.etoilebsd.net> <201208310810.50725.jhb@freebsd.org> <20120831122211.GS64447@ithaqua.etoilebsd.net> <50424956.4090804@freebsd.org> <50426493.7050302@FreeBSD.org> To: Matthew Seaman X-Mailer: Apple Mail (2.1278) Cc: freebsd-current@freebsd.org Subject: Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 00:04:45 -0000 On Sep 1, 2012, at 12:40 PM, Matthew Seaman wrote: > On 01/09/2012 18:43, Tijl Coosemans wrote: >> In this scenario the ports tree needs to keep support for older = releases, >> but that's a consequence of the fact that there's only one ports tree = for >> all releases. Somewhere in between the ports and the various releases = there >> has to be some form encapsulation, not just for pkg, but for all the = tools >> used by the ports tree. Given how the ports tree currently = encapsulates >> both the old and new pkg tools I don't see how supporting multiple = versions >> of pkgng would be a problem because presumably the difference between = pkgng >> versions is going to be much smaller than the difference between the = old >> and new tools. >=20 > New functionality already in the process of development will entail > making non-backwards compatible changes to the DB schema. If we're = tied > to supporting a version of pkgng bundled with a release, that's going = to > make rolling out such changes much harder. On the other hand, if = pkgng > is in ports, then we can release a new version and simultaneously > publish new package sets (incorporating the update to pkgng) from the > repositories which will have been built using the updated DB schema. Will new versions of pkgng support old packages? Some folks maintain their own package repositories and will get rather grumpy if an update to pkgng requires them to rebuild their entire repository. Tim From owner-freebsd-current@FreeBSD.ORG Sun Sep 2 00:27:21 2012 Return-Path: Delivered-To: current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2F891065676 for ; Sun, 2 Sep 2012 00:27:21 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 395268FC0A for ; Sun, 2 Sep 2012 00:27:21 +0000 (UTC) Received: from John-Baldwins-MacBook-Air.local (c-68-39-198-164.hsd1.de.comcast.net [68.39.198.164]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 9FABEB91A for ; Sat, 1 Sep 2012 20:27:20 -0400 (EDT) Message-ID: <5042A7E8.30200@FreeBSD.org> Date: Sat, 01 Sep 2012 20:27:20 -0400 From: John Baldwin User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 To: current@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Sat, 01 Sep 2012 20:27:20 -0400 (EDT) Cc: Subject: [PATCH] Locking for bt(4) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 00:27:21 -0000 I have patches to add locking for the bt(4) driver if anyone is able to test them. The patches should apply to 8 or 9 as well as HEAD. Please enable INVARIANTS during at least initial testing. Thanks. http://www.FreeBSD.org/~jhb/patches/bt_locking.patch -- John Baldwin From owner-freebsd-current@FreeBSD.ORG Sun Sep 2 06:01:37 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF216106566C for ; Sun, 2 Sep 2012 06:01:37 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) by mx1.freebsd.org (Postfix) with ESMTP id 470808FC16 for ; Sun, 2 Sep 2012 06:01:37 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.187.76.163]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.5/8.14.5) with ESMTP id q8261XsC015991 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 2 Sep 2012 07:01:33 +0100 (BST) (envelope-from matthew@FreeBSD.org) X-DKIM: OpenDKIM Filter v2.5.2 smtp.infracaninophile.co.uk q8261XsC015991 Authentication-Results: smtp.infracaninophile.co.uk/q8261XsC015991; dkim=none (no signature); dkim-adsp=none Message-ID: <5042F634.8020104@FreeBSD.org> Date: Sun, 02 Sep 2012 07:01:24 +0100 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 To: Tim Kientzle References: <97612B57-1255-4BB3-A6D3-FC74324C6D67@FreeBSD.org> <503FF0EE.2020605@FreeBSD.org> <20120831095910.GQ64447@ithaqua.etoilebsd.net> <201208310810.50725.jhb@freebsd.org> <20120831122211.GS64447@ithaqua.etoilebsd.net> <50424956.4090804@freebsd.org> <50426493.7050302@FreeBSD.org> <18B2DCFF-3769-46BF-9801-AD06E0A75A40@kientzle.com> In-Reply-To: <18B2DCFF-3769-46BF-9801-AD06E0A75A40@kientzle.com> X-Enigmail-Version: 1.4.4 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigCE5B92E11E7B023C6A996D72" X-Virus-Scanned: clamav-milter 0.97.5 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.7 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk Cc: freebsd-current@FreeBSD.org Subject: Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 06:01:37 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigCE5B92E11E7B023C6A996D72 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 02/09/2012 01:04, Tim Kientzle wrote: > Will new versions of pkgng support old packages? >=20 > Some folks maintain their own package repositories and > will get rather grumpy if an update to pkgng requires them > to rebuild their entire repository. There's a distinction between the format of pkg tarballs, and the formats of the repository catalogue database or the locally installed packages database. If you're maintaining your own repository, then an update to the repo catalogue format means you'ld just need to re-run 'pkg repo'. You won't need to rebuild all the existing package tarballs in your repository. If the repository catalogue format has changed, pkg repo will detect this, and automatically do a full repo catalogue rebuild rather than an incremental one. As rebuilding the repo database is something you'ld do routinely anyhow as part of normal maintenance I don't see this as being a significant extra burden. Similarly, an update to the locally installed packages database schema will be applied transparently when you first use the updated version of pkgng. It won't require you to reinstall any packages. There aren't any plans to change the pkg tarball format that I know of at the moment, but if there were, then they certainly would have to maintain backwards compatibility -- old pkg tarballs will still work with the newer pkgng. Not sure about any guarantees that vice-versa would always work, but way the YAML metadata in the pkg tarball is handled is tolerant of new additions, so it should usually be possible to arrange things so that an older pkgng can cope with a newer pkg tarbal= l. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. PGP: http://www.infracaninophile.co.uk/pgpkey --------------enigCE5B92E11E7B023C6A996D72 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBC9jwACgkQ8Mjk52CukIxHIgCfYN3WLgQepc6z7B5tSnx0hUhn SKgAn3lJ3RgyS19Vjmk8UOB+BdR+29Fq =BuXV -----END PGP SIGNATURE----- --------------enigCE5B92E11E7B023C6A996D72-- From owner-freebsd-current@FreeBSD.ORG Sun Sep 2 07:16:10 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx2.freebsd.org (mx2.freebsd.org [69.147.83.53]) by hub.freebsd.org (Postfix) with ESMTP id C228D106564A; Sun, 2 Sep 2012 07:16:10 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from opti.dougb.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 53B9914F686; Sun, 2 Sep 2012 07:16:10 +0000 (UTC) Message-ID: <504307BA.5050805@FreeBSD.org> Date: Sun, 02 Sep 2012 00:16:10 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120728 Thunderbird/14.0 MIME-Version: 1.0 To: Matthew Seaman References: <97612B57-1255-4BB3-A6D3-FC74324C6D67@FreeBSD.org> <503FF0EE.2020605@FreeBSD.org> <20120831095910.GQ64447@ithaqua.etoilebsd.net> <201208310810.50725.jhb@freebsd.org> <20120831122211.GS64447@ithaqua.etoilebsd.net> <50424956.4090804@freebsd.org> <50426493.7050302@FreeBSD.org> <18B2DCFF-3769-46BF-9801-AD06E0A75A40@kientzle.com> <5042F634.8020104@FreeBSD.org> In-Reply-To: <5042F634.8020104@FreeBSD.org> X-Enigmail-Version: 1.4.3 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-current@FreeBSD.org Subject: Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 07:16:10 -0000 On 09/01/2012 23:01, Matthew Seaman wrote: > As rebuilding the repo database is something you'ld do routinely anyhow > as part of normal maintenance Errr ... what? Why would this be true? Doesn't pkg keep the repo database up to date as it's making changes? -- I am only one, but I am one. I cannot do everything, but I can do something. And I will not let what I cannot do interfere with what I can do. -- Edward Everett Hale, (1822 - 1909) From owner-freebsd-current@FreeBSD.ORG Sun Sep 2 07:18:43 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 2788B1065673; Sun, 2 Sep 2012 07:18:43 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from opti.dougb.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id D754814F135; Sun, 2 Sep 2012 07:18:42 +0000 (UTC) Message-ID: <50430852.7090906@FreeBSD.org> Date: Sun, 02 Sep 2012 00:18:42 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120728 Thunderbird/14.0 MIME-Version: 1.0 To: Garrett Cooper References: <97612B57-1255-4BB3-A6D3-FC74324C6D67@FreeBSD.org> <503FF0EE.2020605@FreeBSD.org> <20120831095910.GQ64447@ithaqua.etoilebsd.net> <201208310810.50725.jhb@freebsd.org> <20120831122211.GS64447@ithaqua.etoilebsd.net> <50424956.4090804@freebsd.org> <50426493.7050302@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.4.3 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, Matthew Seaman Subject: Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 07:18:43 -0000 On 09/01/2012 12:59, Garrett Cooper wrote: > Again, this is part of the reason why I suggested multiple release > trains. Although it's more painful for bapt@, et all, it's ultimately > what would need to be done in order for pkgng to be packaged with a > release or set of releases. Garrett, I think you're seriously misunderstanding how pkg is going to work. One thing we desperately want to get away from is tying ports stuff to specific base releases. Can you please be more clear as to why you keep trying to pull things back in the wrong direction? -- I am only one, but I am one. I cannot do everything, but I can do something. And I will not let what I cannot do interfere with what I can do. -- Edward Everett Hale, (1822 - 1909) From owner-freebsd-current@FreeBSD.ORG Sun Sep 2 07:26:22 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id 45F41106564A; Sun, 2 Sep 2012 07:26:22 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from opti.dougb.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id ECC5514EF89; Sun, 2 Sep 2012 07:26:21 +0000 (UTC) Message-ID: <50430A1D.2050700@FreeBSD.org> Date: Sun, 02 Sep 2012 00:26:21 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120728 Thunderbird/14.0 MIME-Version: 1.0 To: Dimitry Andric References: <503D12CB.4000208@FreeBSD.org> <503D29F4.1030804@FreeBSD.org> <503D35DA.9060704@FreeBSD.org> <503F91C0.3090208@FreeBSD.org> In-Reply-To: <503F91C0.3090208@FreeBSD.org> X-Enigmail-Version: 1.4.3 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Eir Nym , FreeBSD Mail Lists Subject: Re: Can't build FreeBSD-head with CLANG X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 07:26:22 -0000 On 08/30/2012 09:16, Dimitry Andric wrote: > [Note that linking GPL-contaminated code into your > kernel proper is, shall we say, "ideologically impure" ;-) But that is > not the issue here.] Can you keep this kind of stuff to -chat please? The more we deal with the technical aspects the better off we will all be. I for one put ext2fs in my kernel config because I have critical stuff on those file systems and I both do want the speed boost and don't want to worry about what's going to happen when I boot a new kernel. Tools, not policy. Doug -- I am only one, but I am one. I cannot do everything, but I can do something. And I will not let what I cannot do interfere with what I can do. -- Edward Everett Hale, (1822 - 1909) From owner-freebsd-current@FreeBSD.ORG Sun Sep 2 07:37:49 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E9BA6106566C; Sun, 2 Sep 2012 07:37:49 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) by mx1.freebsd.org (Postfix) with ESMTP id 4FA638FC14; Sun, 2 Sep 2012 07:37:49 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.187.76.163]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.5/8.14.5) with ESMTP id q827bjgo017508 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 2 Sep 2012 08:37:45 +0100 (BST) (envelope-from matthew@FreeBSD.org) X-DKIM: OpenDKIM Filter v2.5.2 smtp.infracaninophile.co.uk q827bjgo017508 Authentication-Results: smtp.infracaninophile.co.uk/q827bjgo017508; dkim=none (no signature); dkim-adsp=none Message-ID: <50430CC1.1080608@FreeBSD.org> Date: Sun, 02 Sep 2012 08:37:37 +0100 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 To: Doug Barton References: <97612B57-1255-4BB3-A6D3-FC74324C6D67@FreeBSD.org> <503FF0EE.2020605@FreeBSD.org> <20120831095910.GQ64447@ithaqua.etoilebsd.net> <201208310810.50725.jhb@freebsd.org> <20120831122211.GS64447@ithaqua.etoilebsd.net> <50424956.4090804@freebsd.org> <50426493.7050302@FreeBSD.org> <18B2DCFF-3769-46BF-9801-AD06E0A75A40@kientzle.com> <5042F634.8020104@FreeBSD.org> <504307BA.5050805@FreeBSD.org> In-Reply-To: <504307BA.5050805@FreeBSD.org> X-Enigmail-Version: 1.4.4 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigC7C54E11A8AE0041F1BEAC99" X-Virus-Scanned: clamav-milter 0.97.5 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.7 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk Cc: freebsd-current@FreeBSD.org Subject: Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 07:37:50 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigC7C54E11A8AE0041F1BEAC99 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 02/09/2012 08:16, Doug Barton wrote: > On 09/01/2012 23:01, Matthew Seaman wrote: >> As rebuilding the repo database is something you'ld do routinely anyho= w >> as part of normal maintenance >=20 > Errr ... what? Why would this be true? Doesn't pkg keep the repo > database up to date as it's making changes? Other tools like poudriere or tinderbox are used to maintain the repository -- building ports etc. These tools invoke 'pkg create' to create each individual pkg tarball, and at the end of a session of package building invoke 'pkg repo' one time to update the repository catalogue. It's that last step I was describing. Mind you, having a mode to add a package to the repo and update the catalogue all in one would be pretty useful. Good idea. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. PGP: http://www.infracaninophile.co.uk/pgpkey --------------enigC7C54E11A8AE0041F1BEAC99 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBDDMkACgkQ8Mjk52CukIzJywCdGZsXrwbyTOamfDv5w1Eh5TXO +y0An20pTtakpJvN9+pTgwhmYRAeR5F/ =VKxY -----END PGP SIGNATURE----- --------------enigC7C54E11A8AE0041F1BEAC99-- From owner-freebsd-current@FreeBSD.ORG Sun Sep 2 10:34:11 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3A7C6106566B for ; Sun, 2 Sep 2012 10:34:11 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 926978FC0C for ; Sun, 2 Sep 2012 10:34:09 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q82AYIfW017622 for ; Sun, 2 Sep 2012 13:34:18 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q82AY6mZ024811 for ; Sun, 2 Sep 2012 13:34:06 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q82AY6os024810 for current@freebsd.org; Sun, 2 Sep 2012 13:34:06 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 2 Sep 2012 13:34:06 +0300 From: Konstantin Belousov To: current@freebsd.org Message-ID: <20120902103406.GU33100@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="vX4pGA+I6ZThi/bt" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Subject: Bull Mountain (IvyBridge +) random number generator X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Sep 2012 10:34:11 -0000 --vX4pGA+I6ZThi/bt Content-Type: text/plain; charset=us-ascii Content-Disposition: inline It is relatively well known that Ivy Bridge CPUs (Core iX 3XXX) have built-in hardware random number generator, which is claimed to be both very fast and high quality. Generator is accessible using non-privileged RDRAND instruction. It is claimed that CPU performs sanitization of the random sequence. In particular, it seems that paranoid AES encryption of the raw random stream, performed by our padlock driver, is not needed for Bull Mountain (there are hints that hardware performs it already). See http://spectrum.ieee.org/computing/hardware/behind-intels-new-randomnumber-generator/0 http://software.intel.com/en-us/articles/intel-digital-random-number-generator-drng-software-implementation-guide/ and IA32 ADM. Patch at http://people.freebsd.org/~kib/misc/bull_mountain.2.patch implements support for the generator. I do not own any IvyBridge machines, so I cannot test. Patch makes both padlock and bull generators the options, you need to enable IVY_RNG to get support for the generator. I would be interested in seeing reports including verbose boot dmesg, and some tests of /dev/random quality on the IvyBridge machines, you can start with http://lists.gnupg.org/pipermail/gnupg-devel/2000-March/016328.html. Thanks. --vX4pGA+I6ZThi/bt Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlBDNh4ACgkQC3+MBN1Mb4iDyQCdFEfLX2PL9oGK2wsNPK/m8zAk HkgAoPdlrSbZXf5iBrllCo4rc1vvtM6J =EOI8 -----END PGP SIGNATURE----- --vX4pGA+I6ZThi/bt-- From owner-freebsd-current@FreeBSD.ORG Mon Sep 3 06:49:12 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4183F106567E for ; Mon, 3 Sep 2012 06:49:12 +0000 (UTC) (envelope-from bsam@passap.ru) Received: from forward12.mail.yandex.net (forward12.mail.yandex.net [IPv6:2a02:6b8:0:801::2]) by mx1.freebsd.org (Postfix) with ESMTP id 4CE5A8FC15 for ; Mon, 3 Sep 2012 06:49:11 +0000 (UTC) Received: from smtp14.mail.yandex.net (smtp14.mail.yandex.net [95.108.131.192]) by forward12.mail.yandex.net (Yandex) with ESMTP id D0E56C21D91 for ; Mon, 3 Sep 2012 10:49:09 +0400 (MSK) Received: from smtp14.mail.yandex.net (localhost [127.0.0.1]) by smtp14.mail.yandex.net (Yandex) with ESMTP id BAA7D1B60740 for ; Mon, 3 Sep 2012 10:49:09 +0400 (MSK) Received: from 87.249.28.58.tel.ru (87.249.28.58.tel.ru [87.249.28.58]) by smtp14.mail.yandex.net (nwsmtp/Yandex) with ESMTP id n9ZGLxr5-n9ZGADFW; Mon, 3 Sep 2012 10:49:09 +0400 Message-ID: <504452E5.5040508@passap.ru> Date: Mon, 03 Sep 2012 10:49:09 +0400 From: =?UTF-8?B?0JHQvtGA0LjRgSDQodCw0LzQvtGA0L7QtNC+0LI=?= User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:14.0) Gecko/20120728 Thunderbird/14.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org References: <50407056.3000606@passap.ru> <50407912.60809@passap.ru> <20120901144213.GF3324@home.opsec.eu> In-Reply-To: <20120901144213.GF3324@home.opsec.eu> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Subject: Re: rpcbind does not honor -h flag X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 06:49:12 -0000 01.09.2012 18:42, Kurt Jaeger пишет: > Hi! > >>>>>> Please file a PR against rc ASAP. > >>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/117711 > >> Looks like Matteo Riondato had created a patch for the problem in 2008: >> >> http://people.freebsd.org/~matteo/diff/117711rpcbind.diff >> >> but he never received any feedback from Carlos Eduardo Monti to see if >> the patch fixed the problem. > >> I don't know if the patch will apply to the current FreeBSD rpcbind >> code, give it a try and submit a follow up to the PR. > > In the current form the patch fails in rpcbind.c on 9.1-RC1. > > There are two problems with the current rpcbind.c. > > 1) It seems to be that even if some -h is given, the > rpcbind code uses some SUN-RPC trickery around the /etc/netconfig > file to open sockets for localhost in v4 and v6. > > Is it required to bind to localhost according to the RPC spec ? > > 2) And it opens some dynamic ports for other uses -- anybody has an > idea why this is necessary ? Is there an requirement for this in the spec ? > > Below is an example of both issues. > > root rpcbind 2134 4 udp6 *:* *:* > root rpcbind 2134 5 stream /var/run/rpcbind.sock > root rpcbind 2134 6 udp6 *:111 *:* > root rpcbind 2134 7 udp6 *:924 *:* > root rpcbind 2134 8 tcp6 *:111 *:* > root rpcbind 2134 9 udp4 *:111 *:* > root rpcbind 2134 10 udp4 *:645 *:* > root rpcbind 2134 11 tcp4 *:111 *:* > > Here's rpcbind started with -h : > > root rpcbind 2195 4 udp6 *:* *:* > root rpcbind 2195 5 stream /var/run/rpcbind.sock > root rpcbind 2195 6 udp6 ::1:111 *:* > root rpcbind 2195 7 udp6 *:1013 *:* > root rpcbind 2195 8 tcp6 ::1:111 *:* > root rpcbind 2195 9 udp4 127.0.0.1:111 *:* > root rpcbind 2195 10 udp4 :111 *:* > root rpcbind 2195 11 udp4 *:634 *:* > root rpcbind 2195 12 tcp4 127.0.0.1:111 *:* > root rpcbind 2195 13 tcp4 :111 *:* > > One can see two dynamic udp ports opened (one v4, one v6). > > I might be naive, but from what I understand, it should not open > that many sockets, but only like this: > > root rpcbind 2195 10 udp4 :111 *:* > root rpcbind 2195 13 tcp4 :111 *:* > > If this naive 'spec' is correct, would a patch to do just this and > nothing more be OK ? Patches are always welcome. But please read RPCBIND(8) first. Thanks for your time! -- WBR, Boris Samorodov (bsam) FreeBSD Committer, http://www.FreeBSD.org The Power To Serve From owner-freebsd-current@FreeBSD.ORG Mon Sep 3 10:35:09 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BEB06106564A for ; Mon, 3 Sep 2012 10:35:09 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: from mail-wi0-f178.google.com (mail-wi0-f178.google.com [209.85.212.178]) by mx1.freebsd.org (Postfix) with ESMTP id 4E1D58FC08 for ; Mon, 3 Sep 2012 10:35:09 +0000 (UTC) Received: by wibhr14 with SMTP id hr14so2858869wib.13 for ; Mon, 03 Sep 2012 03:35:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=D4z3D0Ew3wGTh10OicATQc7pH+tFraJHDR3txSPgyT8=; b=vXfnMNCPhi4a1+QiK4FlTOBLO6Yy9P4oLspK89VGxAL1Xgu26JCBNkwm5/PnffSvcX jAwX2psQ537frWKOXpnLESxg0yHq7Fp+ZBgxqQz0Zoppo2DeuaYjXPy5As7ejUoLUJQA t08TdxjjkmzMqFSMse42PgIRwAJ3vOIYIsJ5Q2sgjoA6qAfOyWey9OZnR4fyCrcV3v/R pnbf4kyHVP7a+LXEpGbuhzoTxkmAr+jxpiMHdBBMgtM4VV4uTQHL72MQem+8GOuo/7xj 1yNyOlrrslHomxd2+C1ZhO1/NUS4fevZW0X6Esl6FXPEM/sdtC57G362ritvBTyqPzZ6 ZRyA== MIME-Version: 1.0 Received: by 10.180.93.8 with SMTP id cq8mr22276715wib.16.1346668508180; Mon, 03 Sep 2012 03:35:08 -0700 (PDT) Received: by 10.194.0.145 with HTTP; Mon, 3 Sep 2012 03:35:08 -0700 (PDT) Date: Mon, 3 Sep 2012 12:35:08 +0200 Message-ID: From: Svatopluk Kraus To: freebsd-current@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 10:35:09 -0000 Hi, I found out that while the running excecutables and a dynamic linker are protected against writing (ETXTBSY), the loaded shared libraries are not protected. The libraries are mapped by mmap() in dynamic linker (rtld) and there is no way how to set VV_TEXT flag on the libraries vnodes in mmap() code. In linux compability code \compat\linux\linux_misc.c, linux_uselib() sets VV_TEXT flags on a library vnode. In Solaris, MAP_TEXT flag exists which informs mmap() that the mapped region will be used primarily for executing instructions (for better MMU utilization). With these on mind, I propose to implement MAP_TEXT option in mmap() and in case that underlying object is a vnode, set VV_TEXT flag on it. I already have implemented it and with rtld map_object() patch it works fine for me (of course). The rtld patch looks easy, however I'm not sure about mmap patch. After some investigation, it looks that VV_TEXT once set on a vnode remains set until last reference on the vnode is left. So, I don't bother with VV_TEXT unset in munmap() to be consistent. The executables and dynamic linker are activated in kernel, so VV_TEXT is set before activation and cleared if something failed. Shared library activation is done in dynamic linker (i.e., in userland). It's done in steps and mmaping the library is one from them. So, I think that VV_TEXT can be set in mmap() just after everything is finished successfully. The patch itself is implemented in vm_mmap_vnode(). If I want to set VV_TEXT flag on a vnode, I need an exclusive lock. In current code, the exclusive lock flag is (mis)used as a flag for vnode_pager_update_writecount() call. (I hope that I didn't miss something.) So, the patch is bigger slightly. I defined the MAP_TEXT flag in extented flags sections. However, I'm feeling the relation to MAP_STACK flag, but not sure if and when reserved flags (in other flags section) can be re-used. Svata Index: libexec/rtld-elf/map_object.c =================================================================== --- libexec/rtld-elf/map_object.c (revision 239770) +++ libexec/rtld-elf/map_object.c (working copy) @@ -199,7 +199,8 @@ data_prot = convert_prot(segs[i]->p_flags); data_flags = convert_flags(segs[i]->p_flags) | MAP_FIXED; if (mmap(data_addr, data_vlimit - data_vaddr, data_prot, - data_flags | MAP_PREFAULT_READ, fd, data_offset) == (caddr_t) -1) { + data_flags | MAP_PREFAULT_READ | MAP_TEXT, fd, data_offset) == + (caddr_t) -1) { _rtld_error("%s: mmap of data failed: %s", path, rtld_strerror(errno)); goto error1; Index: sys/vm/vm_mmap.c =================================================================== --- sys/vm/vm_mmap.c (revision 239770) +++ sys/vm/vm_mmap.c (working copy) @@ -1258,10 +1258,13 @@ struct mount *mp; struct ucred *cred; int error, flags, locktype, vfslocked; + int writeable_shared; mp = vp->v_mount; cred = td->td_ucred; - if ((*maxprotp & VM_PROT_WRITE) && (*flagsp & MAP_SHARED)) + flags = *flagsp; + writeable_shared = ((*maxprotp & VM_PROT_WRITE) && (flags & MAP_SHARED)); + if (writeable_shared || ((flags & MAP_TEXT) != 0)) locktype = LK_EXCLUSIVE; else locktype = LK_SHARED; @@ -1271,7 +1274,6 @@ return (error); } foff = *foffp; - flags = *flagsp; obj = vp->v_object; if (vp->v_type == VREG) { /* @@ -1294,7 +1296,7 @@ return (error); } } - if (locktype == LK_EXCLUSIVE) { + if (writeable_shared) { *writecounted = TRUE; vnode_pager_update_writecount(obj, 0, objsize); } @@ -1337,6 +1339,14 @@ error = ENOMEM; goto done; } + /* + * If MAP_TEXT is announced, set VV_TEXT so no one can write + * to the executable. + */ + if ((flags & MAP_TEXT) != 0) { + ASSERT_VOP_ELOCKED(vp, "vv_text"); + vp->v_vflag |= VV_TEXT; + } *objp = obj; *flagsp = flags; Index: sys/sys/mman.h =================================================================== --- sys/sys/mman.h (revision 239770) +++ sys/sys/mman.h (working copy) @@ -91,6 +91,7 @@ */ #define MAP_NOCORE 0x00020000 /* dont include these pages in a coredump */ #define MAP_PREFAULT_READ 0x00040000 /* prefault mapping for reading */ +#define MAP_TEXT 0x00080000 /* map code segment */ #endif /* __BSD_VISIBLE */ #if __POSIX_VISIBLE >= 199309 From owner-freebsd-current@FreeBSD.ORG Mon Sep 3 10:50:59 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09410106566C for ; Mon, 3 Sep 2012 10:50:59 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 2A7A38FC0A for ; Mon, 3 Sep 2012 10:50:57 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA20790; Mon, 03 Sep 2012 13:50:55 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1T8UF5-000KIy-5Q; Mon, 03 Sep 2012 13:50:55 +0300 Message-ID: <50448B8D.7080108@FreeBSD.org> Date: Mon, 03 Sep 2012 13:50:53 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120901 Thunderbird/15.0 MIME-Version: 1.0 To: Svatopluk Kraus References: In-Reply-To: X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-current@FreeBSD.org Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 10:50:59 -0000 on 03/09/2012 13:35 Svatopluk Kraus said the following: > After some investigation, it looks that VV_TEXT once set on a vnode > remains set until last reference on the vnode is left. There was an idea to turn VV_TEXT flag into a v_text counter. Maybe something like that would be useful indeed? -- Andriy Gapon From owner-freebsd-current@FreeBSD.ORG Mon Sep 3 12:04:25 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6CA6C1065673 for ; Mon, 3 Sep 2012 12:04:25 +0000 (UTC) (envelope-from fabien.thomas@netasq.com) Received: from work.netasq.com (gwlille.netasq.com [91.212.116.1]) by mx1.freebsd.org (Postfix) with ESMTP id 0787D8FC1E for ; Mon, 3 Sep 2012 12:04:24 +0000 (UTC) Received: from [10.2.1.1] (unknown [10.2.1.1]) by work.netasq.com (Postfix) with ESMTPSA id DEAC327041D4 for ; Mon, 3 Sep 2012 14:04:23 +0200 (CEST) From: Fabien Thomas Content-Type: multipart/signed; boundary="Apple-Mail=_D2EF3AD3-818E-4F0D-B91F-EA6042A7888C"; protocol="application/pkcs7-signature"; micalg=sha1 Date: Mon, 3 Sep 2012 14:04:21 +0200 Message-Id: <0D822C75-7240-42E3-8780-18CABC9C45A1@netasq.com> To: freebsd-current Mime-Version: 1.0 (Apple Message framework v1278) X-Mailer: Apple Mail (2.1278) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: [CFT] hwpmc support for Intel Ivy Bridge X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 12:04:25 -0000 --Apple-Mail=_D2EF3AD3-818E-4F0D-B91F-EA6042A7888C Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii Hi, Find a patch that add Intel Ivy Bridge support to hwpmc(9). The patch also support offcore RSP token for Sandy Bridge. Note: No uncore support. Tested on: Intel(R) Xeon(R) CPU E3-1265L V2 @ 2.50GHz (2494.35-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x306a9 Family = 6 Model = 3a Stepping = 9 http://people.freebsd.org/~fabient/patch-hwpmc_ivy_bridge_head cd /usr/src && patch -p0 < patch-hwpmc_ivy_bridge_head and rebuild world. Fabien --Apple-Mail=_D2EF3AD3-818E-4F0D-B91F-EA6042A7888C-- From owner-freebsd-current@FreeBSD.ORG Mon Sep 3 12:46:51 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2D07E106566B for ; Mon, 3 Sep 2012 12:46:51 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 71F228FC08 for ; Mon, 3 Sep 2012 12:46:49 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q83CktIi053458; Mon, 3 Sep 2012 15:46:55 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q83Ckg24032916; Mon, 3 Sep 2012 15:46:42 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q83CkgJi032915; Mon, 3 Sep 2012 15:46:42 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 3 Sep 2012 15:46:42 +0300 From: Konstantin Belousov To: Svatopluk Kraus Message-ID: <20120903124642.GI33100@deviant.kiev.zoral.com.ua> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="q+qpHfi2NROeSQHU" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-current@freebsd.org Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 12:46:51 -0000 --q+qpHfi2NROeSQHU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Sep 03, 2012 at 12:35:08PM +0200, Svatopluk Kraus wrote: > Hi, >=20 > I found out that while the running excecutables and a dynamic linker > are protected against writing (ETXTBSY), the loaded shared libraries > are not protected. The libraries are mapped by mmap() in dynamic > linker (rtld) and there is no way how to set VV_TEXT flag on the > libraries vnodes in mmap() code. >=20 > In linux compability code \compat\linux\linux_misc.c, linux_uselib() > sets VV_TEXT flags on a library vnode. In Solaris, MAP_TEXT flag > exists which informs mmap() that the mapped region will be used > primarily for executing instructions (for better MMU utilization). > With these on mind, I propose to implement MAP_TEXT option in mmap() > and in case that underlying object is a vnode, set VV_TEXT flag on it. >=20 > I already have implemented it and with rtld map_object() patch it > works fine for me (of course). The rtld patch looks easy, however I'm > not sure about mmap patch. >=20 > After some investigation, it looks that VV_TEXT once set on a vnode > remains set until last reference on the vnode is left. So, I don't > bother with VV_TEXT unset in munmap() to be consistent. The > executables and dynamic linker are activated in kernel, so VV_TEXT is > set before activation and cleared if something failed. Shared library > activation is done in dynamic linker (i.e., in userland). It's done in > steps and mmaping the library is one from them. So, I think that > VV_TEXT can be set in mmap() just after everything is finished > successfully. This is right, the object reference counter is also used as VV_TEXT counter. It is somewhat unaccurate, but in practice does not cause issues. >=20 > The patch itself is implemented in vm_mmap_vnode(). If I want to set > VV_TEXT flag on a vnode, I need an exclusive lock. In current code, > the exclusive lock flag is (mis)used as a flag for > vnode_pager_update_writecount() call. (I hope that I didn't miss > something.) So, the patch is bigger slightly. >=20 > I defined the MAP_TEXT flag in extented flags sections. However, I'm > feeling the relation to MAP_STACK flag, but not sure if and when > reserved flags (in other flags section) can be re-used. >=20 > Svata >=20 >=20 > Index: libexec/rtld-elf/map_object.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- libexec/rtld-elf/map_object.c (revision 239770) > +++ libexec/rtld-elf/map_object.c (working copy) > @@ -199,7 +199,8 @@ > data_prot =3D convert_prot(segs[i]->p_flags); > data_flags =3D convert_flags(segs[i]->p_flags) | MAP_FIXED; > if (mmap(data_addr, data_vlimit - data_vaddr, data_prot, > - data_flags | MAP_PREFAULT_READ, fd, data_offset) =3D=3D (caddr_t) -1)= { > + data_flags | MAP_PREFAULT_READ | MAP_TEXT, fd, data_offset) =3D=3D > + (caddr_t) -1) { I am not sure that we shall mark all segments mappings with MAP_TEXT. I understand the logic of the change, since we do not want data segment to be changed under us. Still, having MAP_TEXT for non-text segments looks strange. > _rtld_error("%s: mmap of data failed: %s", path, > rtld_strerror(errno)); > goto error1; > Index: sys/vm/vm_mmap.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/vm/vm_mmap.c (revision 239770) > +++ sys/vm/vm_mmap.c (working copy) > @@ -1258,10 +1258,13 @@ > struct mount *mp; > struct ucred *cred; > int error, flags, locktype, vfslocked; > + int writeable_shared; >=20 > mp =3D vp->v_mount; > cred =3D td->td_ucred; > - if ((*maxprotp & VM_PROT_WRITE) && (*flagsp & MAP_SHARED)) > + flags =3D *flagsp; > + writeable_shared =3D ((*maxprotp & VM_PROT_WRITE) && (flags & MAP_SHARE= D)); > + if (writeable_shared || ((flags & MAP_TEXT) !=3D 0)) > locktype =3D LK_EXCLUSIVE; > else > locktype =3D LK_SHARED; > @@ -1271,7 +1274,6 @@ > return (error); > } > foff =3D *foffp; > - flags =3D *flagsp; > obj =3D vp->v_object; > if (vp->v_type =3D=3D VREG) { > /* > @@ -1294,7 +1296,7 @@ > return (error); > } > } > - if (locktype =3D=3D LK_EXCLUSIVE) { > + if (writeable_shared) { > *writecounted =3D TRUE; > vnode_pager_update_writecount(obj, 0, objsize); > } > @@ -1337,6 +1339,14 @@ > error =3D ENOMEM; > goto done; > } > + /* > + * If MAP_TEXT is announced, set VV_TEXT so no one can write > + * to the executable. > + */ > + if ((flags & MAP_TEXT) !=3D 0) { > + ASSERT_VOP_ELOCKED(vp, "vv_text"); > + vp->v_vflag |=3D VV_TEXT; > + } I do not think we want to set VV_TEXT for device vnodes. > *objp =3D obj; > *flagsp =3D flags; >=20 > Index: sys/sys/mman.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/sys/mman.h (revision 239770) > +++ sys/sys/mman.h (working copy) > @@ -91,6 +91,7 @@ > */ > #define MAP_NOCORE 0x00020000 /* dont include these pages in a coredump= */ > #define MAP_PREFAULT_READ 0x00040000 /* prefault mapping for reading */ > +#define MAP_TEXT 0x00080000 /* map code segment */ > #endif /* __BSD_VISIBLE */ >=20 > #if __POSIX_VISIBLE >=3D 199309 > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" --q+qpHfi2NROeSQHU Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlBEprIACgkQC3+MBN1Mb4h5yACggbaWE+xAW3fsF8IXMohsHxtW pZwAmwb4i4Mgt/RHBLdM56PNcE3Qsxdk =a2Zn -----END PGP SIGNATURE----- --q+qpHfi2NROeSQHU-- From owner-freebsd-current@FreeBSD.ORG Mon Sep 3 19:05:02 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28A14106566B; Mon, 3 Sep 2012 19:05:02 +0000 (UTC) (envelope-from tinderbox@freebsd.org) Received: from freebsd-current.sentex.ca (freebsd-current.sentex.ca [64.7.128.98]) by mx1.freebsd.org (Postfix) with ESMTP id CEF638FC17; Mon, 3 Sep 2012 19:05:01 +0000 (UTC) Received: from freebsd-current.sentex.ca (localhost [127.0.0.1]) by freebsd-current.sentex.ca (8.14.5/8.14.5) with ESMTP id q83IwItp098339; Mon, 3 Sep 2012 14:58:18 -0400 (EDT) (envelope-from tinderbox@freebsd.org) Received: (from tinderbox@localhost) by freebsd-current.sentex.ca (8.14.5/8.14.5/Submit) id q83IwI2x098317; Mon, 3 Sep 2012 18:58:18 GMT (envelope-from tinderbox@freebsd.org) Date: Mon, 3 Sep 2012 18:58:18 GMT Message-Id: <201209031858.q83IwI2x098317@freebsd-current.sentex.ca> X-Authentication-Warning: freebsd-current.sentex.ca: tinderbox set sender to FreeBSD Tinderbox using -f Sender: FreeBSD Tinderbox From: FreeBSD Tinderbox To: FreeBSD Tinderbox , , Precedence: bulk Cc: Subject: [head tinderbox] failure on sparc64/sparc64 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 19:05:02 -0000 TB --- 2012-09-03 18:01:51 - tinderbox 2.9 running on freebsd-current.sentex.ca TB --- 2012-09-03 18:01:51 - FreeBSD freebsd-current.sentex.ca 8.3-PRERELEASE FreeBSD 8.3-PRERELEASE #0: Mon Mar 26 13:54:12 EDT 2012 des@freebsd-current.sentex.ca:/usr/obj/usr/src/sys/GENERIC amd64 TB --- 2012-09-03 18:01:51 - starting HEAD tinderbox run for sparc64/sparc64 TB --- 2012-09-03 18:01:51 - cleaning the object tree TB --- 2012-09-03 18:01:51 - cvsupping the source tree TB --- 2012-09-03 18:01:51 - /usr/bin/csup -z -r 3 -g -L 1 -h cvsup.sentex.ca /tinderbox/HEAD/sparc64/sparc64/supfile TB --- 2012-09-03 18:03:28 - building world TB --- 2012-09-03 18:03:28 - CROSS_BUILD_TESTING=YES TB --- 2012-09-03 18:03:28 - MAKEOBJDIRPREFIX=/obj TB --- 2012-09-03 18:03:28 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2012-09-03 18:03:28 - SRCCONF=/dev/null TB --- 2012-09-03 18:03:28 - TARGET=sparc64 TB --- 2012-09-03 18:03:28 - TARGET_ARCH=sparc64 TB --- 2012-09-03 18:03:28 - TZ=UTC TB --- 2012-09-03 18:03:28 - __MAKE_CONF=/dev/null TB --- 2012-09-03 18:03:28 - cd /src TB --- 2012-09-03 18:03:28 - /usr/bin/make -B buildworld >>> World build started on Mon Sep 3 18:03:29 UTC 2012 >>> Rebuilding the temporary build tree >>> stage 1.1: legacy release compatibility shims >>> stage 1.2: bootstrap tools >>> stage 2.1: cleaning up the object tree >>> stage 2.2: rebuilding the object tree >>> stage 2.3: build tools >>> stage 3: cross tools >>> stage 4.1: building includes >>> stage 4.2: building libraries >>> stage 4.3: make dependencies >>> stage 4.4: building everything [...] /src/secure/usr.bin/ssh/../../../crypto/openssh/mux.c:758: error: 'Forward' has no member named 'handle' /src/secure/usr.bin/ssh/../../../crypto/openssh/mux.c: In function 'process_mux_close_fwd': /src/secure/usr.bin/ssh/../../../crypto/openssh/mux.c:854: error: void value not ignored as it ought to be /src/secure/usr.bin/ssh/../../../crypto/openssh/mux.c:858: warning: implicit declaration of function 'channel_cancel_lport_listener' /src/secure/usr.bin/ssh/../../../crypto/openssh/mux.c: In function 'muxclient': /src/secure/usr.bin/ssh/../../../crypto/openssh/mux.c:2104: error: 'SSHMUX_COMMAND_CANCEL_FWD' undeclared (first use in this function) /src/secure/usr.bin/ssh/../../../crypto/openssh/mux.c:2104: error: (Each undeclared identifier is reported only once /src/secure/usr.bin/ssh/../../../crypto/openssh/mux.c:2104: error: for each function it appears in.) *** Error code 1 Stop in /src/secure/usr.bin/ssh. *** Error code 1 Stop in /src/secure/usr.bin. *** Error code 1 Stop in /src/secure. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. TB --- 2012-09-03 18:58:17 - WARNING: /usr/bin/make returned exit code 1 TB --- 2012-09-03 18:58:17 - ERROR: failed to build world TB --- 2012-09-03 18:58:17 - 2620.73 user 477.97 system 3386.84 real http://tinderbox.freebsd.org/tinderbox-head-HEAD-sparc64-sparc64.full From owner-freebsd-current@FreeBSD.ORG Mon Sep 3 19:37:18 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0DAAE106566C; Mon, 3 Sep 2012 19:37:18 +0000 (UTC) (envelope-from tijl@freebsd.org) Received: from mailrelay005.isp.belgacom.be (mailrelay005.isp.belgacom.be [195.238.6.171]) by mx1.freebsd.org (Postfix) with ESMTP id 296E28FC1D; Mon, 3 Sep 2012 19:37:16 +0000 (UTC) X-Belgacom-Dynamic: yes X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EAFUGRVBbsRjw/2dsb2JhbABFuyOBCIIgAQEEAVYjBQsLGAklDwIoHhMBBwEBiAMKujaLDYcjA45igSCVcYJl Received: from 240.24-177-91.adsl-dyn.isp.belgacom.be (HELO kalimero.tijl.coosemans.org) ([91.177.24.240]) by relay.skynet.be with ESMTP; 03 Sep 2012 21:35:48 +0200 Received: from kalimero.tijl.coosemans.org (kalimero.tijl.coosemans.org [127.0.0.1]) by kalimero.tijl.coosemans.org (8.14.5/8.14.5) with ESMTP id q83JZlqe065036; Mon, 3 Sep 2012 21:35:48 +0200 (CEST) (envelope-from tijl@freebsd.org) Message-ID: <5045068C.80200@freebsd.org> Date: Mon, 03 Sep 2012 21:35:40 +0200 From: Tijl Coosemans User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:14.0) Gecko/20120804 Thunderbird/14.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org References: <97612B57-1255-4BB3-A6D3-FC74324C6D67@FreeBSD.org> <503FF0EE.2020605@FreeBSD.org> <20120831095910.GQ64447@ithaqua.etoilebsd.net> <201208310810.50725.jhb@freebsd.org> <20120831122211.GS64447@ithaqua.etoilebsd.net> <50424956.4090804@freebsd.org> <50426493.7050302@FreeBSD.org> In-Reply-To: <50426493.7050302@FreeBSD.org> X-Enigmail-Version: 1.4.2 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------enigF4A976CDEB9CDFD6F10F8CA3" Cc: Matthew Seaman Subject: Re: pkgng suggestion: renaming /usr/sbin/pkg to /usr/sbin/pkg-bootstrap X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 19:37:18 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigF4A976CDEB9CDFD6F10F8CA3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 01-09-2012 21:40, Matthew Seaman wrote: > On 01/09/2012 18:43, Tijl Coosemans wrote: >> In this scenario the ports tree needs to keep support for older releas= es, >> but that's a consequence of the fact that there's only one ports tree = for >> all releases. Somewhere in between the ports and the various releases = there >> has to be some form encapsulation, not just for pkg, but for all the t= ools >> used by the ports tree. Given how the ports tree currently encapsulate= s >> both the old and new pkg tools I don't see how supporting multiple ver= sions >> of pkgng would be a problem because presumably the difference between = pkgng >> versions is going to be much smaller than the difference between the o= ld >> and new tools. >=20 > New functionality already in the process of development will entail > making non-backwards compatible changes to the DB schema. If we're tie= d > to supporting a version of pkgng bundled with a release, that's going t= o > make rolling out such changes much harder. >=20 > On the other hand, if pkgng is in ports, then we can release a new > version and simultaneously publish new package sets (incorporating > the update to pkgng) from the repositories which will have been built > using the updated DB schema. But you cannot update the pkgng repo on the release DVDs. And also, there's no such thing as simultaneous. After you've updated the= port it takes days even weeks for the package build cluster to rebuild package sets for all branches and all architectures (think powerpc, sparc64) and then it takes even more time for the ftp mirrors to pick up the new set from the master ftp and it takes even more time for a user to= actually update his ports/packages (months to years). During all this time there can be a difference in version (possibly several versions) between the pkgng in ports, the pkgng of the official repositories and the pkgng version that is currently installed on the user's system. > The ports tree doesn't track the versioning of the base system, and it > makes perfect sense to me that tools for dealing with the ports should > follow changes to ports rather than changes to the base. How about the following: If you can guarantee that the pkg port can always be built and installed from ports no matter what version of pkg is currently installed, then the= ports tree only needs to support the version of pkg that is currently in the tree. Guaranteeing that the pkg port can always be built probably means it should set a flag in its Makefile (e.g.BUILDING_PKG) that causes bsd.port.mk to not use pkg at all until after installation (so it cannot do conflicts checking for instance). During installation the port also updates the local pkg registry such that after installation bsd.port.mk can register the package with the new pkg version. Special-casing the pkg port this way effectively creates a bootstrap in the ports tree itself (instead of having a pkg-bootstrap tool in base or during FreeBSD installation). Similarly, for package users, pkg should always be able to update itself from a remote repo no matter what version is currently installed. jhb's idea of putting pkg in a self extracting script in a fixed location of a package repo is probably the most flexible. This creates a bootstrap in every pkg repo. And then you can put pkg with a pkg repo on FreeBSD release media as well, such that packages (including pkg) can be installed during and after installation without needing an internet connection. If there is an internet connection and the user wants to install packages from a remote repo, the pkg on the release media can fetch and install pkg from the repo and then that pkg can be used to install other packages. I'd be ok with this. The ports tree only has to support one version of pkgng, there's no separate bootstrap tool, release media are nicely self-contained and no matter how outdated a user's installed packages are he can always update using either ports or packages. --------------enigF4A976CDEB9CDFD6F10F8CA3 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iF4EAREIAAYFAlBFBpMACgkQfoCS2CCgtitrvwD9G6Cbadup6mWc2TfQIofBOb+o uQ54qetj+7+4sI/wFgMA/jcxG+Ot4A+pzdt+k9HanmTqwX/y6iZxSvTM1IyKK1X2 =Z1KX -----END PGP SIGNATURE----- --------------enigF4A976CDEB9CDFD6F10F8CA3-- From owner-freebsd-current@FreeBSD.ORG Mon Sep 3 21:23:44 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3CE49106564A for ; Mon, 3 Sep 2012 21:23:44 +0000 (UTC) (envelope-from bapt@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 205B18FC17; Mon, 3 Sep 2012 21:23:44 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q83LNhDK084235; Mon, 3 Sep 2012 21:23:44 GMT (envelope-from bapt@FreeBSD.org) Received: (from bapt@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q83LNhk3084171; Mon, 3 Sep 2012 21:23:43 GMT (envelope-from bapt@FreeBSD.org) X-Authentication-Warning: freefall.freebsd.org: bapt set sender to bapt@FreeBSD.org using -f Date: Mon, 3 Sep 2012 23:23:40 +0200 From: Baptiste Daroussin To: Fabien Thomas Message-ID: <20120903212340.GE88081@ithaqua.etoilebsd.net> References: <0D822C75-7240-42E3-8780-18CABC9C45A1@netasq.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+JUInw4efm7IfTNU" Content-Disposition: inline In-Reply-To: <0D822C75-7240-42E3-8780-18CABC9C45A1@netasq.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-current Subject: Re: [CFT] hwpmc support for Intel Ivy Bridge X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Sep 2012 21:23:44 -0000 --+JUInw4efm7IfTNU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Sep 03, 2012 at 02:04:21PM +0200, Fabien Thomas wrote: > Hi, >=20 > Find a patch that add Intel Ivy Bridge support to hwpmc(9). > The patch also support offcore RSP token for Sandy Bridge. > Note: No uncore support. >=20 > Tested on: > Intel(R) Xeon(R) CPU E3-1265L V2 @ 2.50GHz (2494.35-MHz K8-class CPU) > Origin =3D "GenuineIntel" Id =3D 0x306a9 Family =3D 6 Model =3D 3a St= epping =3D 9 >=20 > http://people.freebsd.org/~fabient/patch-hwpmc_ivy_bridge_head >=20 > cd /usr/src && patch -p0 < patch-hwpmc_ivy_bridge_head=20 > and rebuild world. >=20 >=20 > Fabien World building, I'll keep you in touch Is there any particulat testing, I can do other than using the new world? regards, Bapt --+JUInw4efm7IfTNU Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlBFH9wACgkQ8kTtMUmk6Ex8JQCaA8S3f4IzJQsu8fcl5zZtCPY1 78YAn166+Vf8EUcgKJgzh0rMo0/V8qpL =oaLc -----END PGP SIGNATURE----- --+JUInw4efm7IfTNU-- From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 03:14:42 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9853A106566B for ; Tue, 4 Sep 2012 03:14:42 +0000 (UTC) (envelope-from andy@neu.net) Received: from mail.neu.net (unknown [IPv6:2607:fc50:0:d300:216:3eff:fe54:f1c6]) by mx1.freebsd.org (Postfix) with ESMTP id 3E40B8FC08 for ; Tue, 4 Sep 2012 03:14:42 +0000 (UTC) Received: from neu.net (neu.net [199.48.129.194]) by mail.neu.net (8.14.5/8.14.5) with ESMTP id q847Egs4058766 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 4 Sep 2012 03:14:42 -0400 (EDT) (envelope-from andy@neu.net) Date: Tue, 4 Sep 2012 03:14:42 -0400 (EDT) From: AN To: freebsd-current@freebsd.org Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Virus-Scanned: clamav-milter 0.97.5 at my.mail.server X-Virus-Status: Clean X-Spam-Status: No, score=0.0 required=4.5 tests=RP_MATCHES_RCVD autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mail.neu.net Subject: recent update seems to break portupgrade X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 03:14:42 -0000 FreeBSD FBSD10 10.0-CURRENT FreeBSD 10.0-CURRENT #18 r240078: Mon Sep 3 17:41:46 EDT 2012 root@FBSD10:/usr/obj/usr/src/sys/MYKERNEL amd64 # cat /etc/make.conf OVERRIDE_LINUX_BASE_PORT=f10 QT4_OPTIONS= QGTKSTYLE # added by use.perl 2012-04-04 01:11:13 PERL_VERSION=5.14.2 MALLOC_PRODUCTION=yes WITH_PKGNG=no I just tried to do a portupgrade and received the following on the command line. I compiled my system (world and ports from source). Portupgrade worked earlier today before I upgraded to r240078. # portupgrade -va ---> Session started at: Mon, 03 Sep 2012 22:51:25 -0400 "/usr/ports/Mk/bsd.openssl.mk", line 109: warning: "/usr/local/sbin/pkg which -qo /usr/local/lib/libcrypto.so" returned non-zero status "/usr/ports/Mk/bsd.openssl.mk", line 109: warning: "/usr/local/sbin/pkg which -qo /usr/local/lib/libcrypto.so" returned non-zero status "/usr/ports/Mk/bsd.openssl.mk", line 109: warning: "/usr/local/sbin/pkg which -qo /usr/local/lib/libcrypto.so" returned non-zero status "/usr/ports/Mk/bsd.openssl.mk", line 109: warning: "/usr/local/sbin/pkg which -qo /usr/local/lib/libcrypto.so" returned non-zero status "/usr/ports/Mk/bsd.openssl.mk", line 109: warning: "/usr/local/sbin/pkg which -qo /usr/local/lib/libcrypto.so" returned non-zero status ** Port directory not found: www/ff_nightly "/usr/ports/Mk/bsd.openssl.mk", line 109: warning: "/usr/local/sbin/pkg which -qo /usr/local/lib/libcrypto.so" returned non-zero status "/usr/ports/Mk/bsd.openssl.mk", line 109: warning: "/usr/local/sbin/pkg which -qo /usr/local/lib/libcrypto.so" returned non-zero status ^C Not sure what is going on, I guess it is related to pkg: # whereis pkg pkg: /usr/sbin/pkg /usr/local/man/man8/pkg.8.gz /usr/src/usr.sbin/pkg [root@FBSD10 ~]# ls -l /usr/sbin/pkg -r-xr-xr-x 1 root wheel 16256 Sep 3 13:55 /usr/sbin/pkg If I do not want to use pkg do I need an entry in /etc/src.conf? How do you disable pkg? How do I run portupgrade now? From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 03:45:12 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E5E311065670 for ; Tue, 4 Sep 2012 03:45:12 +0000 (UTC) (envelope-from bryan@shatow.net) Received: from secure.xzibition.com (secure.xzibition.com [173.160.118.92]) by mx1.freebsd.org (Postfix) with ESMTP id 8D1E88FC1A for ; Tue, 4 Sep 2012 03:45:12 +0000 (UTC) DomainKey-Signature: a=rsa-sha1; c=nofws; d=shatow.net; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; q=dns; s=sweb; b=FTmleD W24DgiwM71zsBbsPUGUUTjtMezScAkm9HRkEc8V3n/xdNuKp+O55x/1l1x5FlQjz xklOmbz5dTiojdIHIlniya657aKsSAPq4e1DSaCm6Vr4P9K8VHTfToDK6xyP1hi5 08gPnJDGh+ehlTm/rC/UlIcWcqiFcna7hxxHU= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=shatow.net; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; s=sweb; bh=jC90imSbPw3z bMy9BsrX7dfmiUNcZbkXKTBzr5QxRFM=; b=Ldrv3I1yw2GUWqIXjEXd3HM72ra8 ZhDBGJnCQ7umnexHyzgQ+9jBPQMSA2vf5dfSy194fGzm42T2aEc1VjHhNg04o3BH aHoiICD5DoDdY6mqED0dVCw/L17opDtFf+f3wQ5RkV3rRwAkOXhyvdcg2jYJPtqD /JI90RbhmxFCDtg= Received: (qmail 73835 invoked from network); 3 Sep 2012 22:45:10 -0500 Received: from unknown (HELO ?10.10.0.115?) (bryan@shatow.net@10.10.0.115) by sweb.xzibition.com with ESMTPA; 3 Sep 2012 22:45:10 -0500 Message-ID: <50457942.90909@shatow.net> Date: Mon, 03 Sep 2012 22:45:06 -0500 From: Bryan Drewery User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 To: AN References: In-Reply-To: X-Enigmail-Version: 1.4.4 OpenPGP: id=3C9B0CF9; url=http://www.shatow.net/bryan/bryan.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Baptiste Daroussin , freebsd-current@freebsd.org, Ports FreeBSD Subject: Re: recent update seems to break portupgrade X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 03:45:13 -0000 On 9/4/2012 2:14 AM, AN wrote: > FreeBSD FBSD10 10.0-CURRENT FreeBSD 10.0-CURRENT #18 r240078: Mon Sep 3 > 17:41:46 EDT 2012 root@FBSD10:/usr/obj/usr/src/sys/MYKERNEL amd64 > > # cat /etc/make.conf > OVERRIDE_LINUX_BASE_PORT=f10 > QT4_OPTIONS= QGTKSTYLE > > # added by use.perl 2012-04-04 01:11:13 > PERL_VERSION=5.14.2 > MALLOC_PRODUCTION=yes > WITH_PKGNG=no *Defining* WITH_PKGNG enables it. This is very common with ports infrastructure. Remove this from your make.conf to not use pkgng. > > I just tried to do a portupgrade and received the following on the > command line. I compiled my system (world and ports from source). > Portupgrade worked earlier today before I upgraded to r240078. > > # portupgrade -va > ---> Session started at: Mon, 03 Sep 2012 22:51:25 -0400 > "/usr/ports/Mk/bsd.openssl.mk", line 109: warning: "/usr/local/sbin/pkg > which -qo /usr/local/lib/libcrypto.so" returned non-zero status > "/usr/ports/Mk/bsd.openssl.mk", line 109: warning: "/usr/local/sbin/pkg > which -qo /usr/local/lib/libcrypto.so" returned non-zero status > "/usr/ports/Mk/bsd.openssl.mk", line 109: warning: "/usr/local/sbin/pkg > which -qo /usr/local/lib/libcrypto.so" returned non-zero status > "/usr/ports/Mk/bsd.openssl.mk", line 109: warning: "/usr/local/sbin/pkg > which -qo /usr/local/lib/libcrypto.so" returned non-zero status > "/usr/ports/Mk/bsd.openssl.mk", line 109: warning: "/usr/local/sbin/pkg > which -qo /usr/local/lib/libcrypto.so" returned non-zero status > ** Port directory not found: www/ff_nightly > "/usr/ports/Mk/bsd.openssl.mk", line 109: warning: "/usr/local/sbin/pkg > which -qo /usr/local/lib/libcrypto.so" returned non-zero status > "/usr/ports/Mk/bsd.openssl.mk", line 109: warning: "/usr/local/sbin/pkg > which -qo /usr/local/lib/libcrypto.so" returned non-zero status This is not a portupgrade problem. These errors are a problem with bsd.port.mk. I've been meaning to write up a patch for a while now and get to bapt for testing. If he doesn't get to it first, I'll get one submitted soon. > ^C > > Not sure what is going on, I guess it is related to pkg: > > # whereis pkg > pkg: /usr/sbin/pkg /usr/local/man/man8/pkg.8.gz /usr/src/usr.sbin/pkg > [root@FBSD10 ~]# ls -l /usr/sbin/pkg > -r-xr-xr-x 1 root wheel 16256 Sep 3 13:55 /usr/sbin/pkg > > If I do not want to use pkg do I need an entry in /etc/src.conf? How do > you disable pkg? How do I run portupgrade now? > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" -- Regards, Bryan Drewery bdrewery@freenode/EFNet From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 07:35:11 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4965C1065675; Tue, 4 Sep 2012 07:35:11 +0000 (UTC) (envelope-from fabien.thomas@netasq.com) Received: from work.netasq.com (gwlille.netasq.com [91.212.116.1]) by mx1.freebsd.org (Postfix) with ESMTP id 3A9288FC12; Tue, 4 Sep 2012 07:35:10 +0000 (UTC) Received: from [10.2.1.1] (unknown [10.2.1.1]) by work.netasq.com (Postfix) with ESMTPSA id 111C9270420F; Tue, 4 Sep 2012 09:35:09 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: multipart/signed; boundary="Apple-Mail=_78EA6D67-D133-4FFF-B040-990090390179"; protocol="application/pkcs7-signature"; micalg=sha1 From: Fabien Thomas In-Reply-To: <20120903212340.GE88081@ithaqua.etoilebsd.net> Date: Tue, 4 Sep 2012 09:35:07 +0200 Message-Id: <60A32BCE-756A-4A0D-A24D-7AC42F71BD1A@netasq.com> References: <0D822C75-7240-42E3-8780-18CABC9C45A1@netasq.com> <20120903212340.GE88081@ithaqua.etoilebsd.net> To: Baptiste Daroussin X-Mailer: Apple Mail (2.1278) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-current Subject: Re: [CFT] hwpmc support for Intel Ivy Bridge X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 07:35:11 -0000 --Apple-Mail=_78EA6D67-D133-4FFF-B040-990090390179 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Le 3 sept. 2012 =E0 23:23, Baptiste Daroussin a =E9crit : > On Mon, Sep 03, 2012 at 02:04:21PM +0200, Fabien Thomas wrote: >> Hi, >>=20 >> Find a patch that add Intel Ivy Bridge support to hwpmc(9). >> The patch also support offcore RSP token for Sandy Bridge. >> Note: No uncore support. >>=20 >> Tested on: >> Intel(R) Xeon(R) CPU E3-1265L V2 @ 2.50GHz (2494.35-MHz K8-class CPU) >> Origin =3D "GenuineIntel" Id =3D 0x306a9 Family =3D 6 Model =3D 3a = Stepping =3D 9 >>=20 >> http://people.freebsd.org/~fabient/patch-hwpmc_ivy_bridge_head >>=20 >> cd /usr/src && patch -p0 < patch-hwpmc_ivy_bridge_head=20 >> and rebuild world. >>=20 >>=20 >> Fabien >=20 > World building, I'll keep you in touch >=20 > Is there any particulat testing, I can do other than using the new = world? Thanks for testing! Yes you can test it quickly using top mode for example: kldload hwpmc pmccontrol -L to list all events some test: - offcore pmcstat -SOFF_CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY -w1 -T - unhalted cycle (programmable counter) pmcstat -n2000000 -SCPU_CLK_UNHALTED.THREAD_P -w4 -T - INSTR_RETIRED_ANY (fixed counter) pmcstat -n2000000 -S INSTR_RETIRED_ANY -w4 -T - Soft counter: pmcstat -SPAGE_FAULT.ALL -w4 -T all at the same time (and change view by pressing 'p': pmcstat -SOFF_CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY -n2000000 = -SCPU_CLK_UNHALTED.THREAD_P -n2000000 -S INSTR_RETIRED_ANY = -SPAGE_FAULT.ALL -w1 -T all events are described in the man page pmc.ivybridge >=20 > regards, > Bapt --Apple-Mail=_78EA6D67-D133-4FFF-B040-990090390179-- From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 12:49:09 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 61A83106566C for ; Tue, 4 Sep 2012 12:49:09 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id E39CA8FC15 for ; Tue, 4 Sep 2012 12:49:08 +0000 (UTC) Received: by weyx56 with SMTP id x56so4358013wey.13 for ; Tue, 04 Sep 2012 05:49:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=W6LYSQ8qXqGRApVPyVhNKZ9cCVY1R+QU46Ku1sPMxXY=; b=qPsur/A2WvcOtFzEJGKW9TIxUtmRtmVz2Q3OqaNpMgrBirxcK9FE1o4dg8YlDu40LC W1JnvlSH4lT6voy8+I6NHyCy4Wnay33Wv/301L2T15G6SYBDDLghTxSeeruPj/QbT/TV qvmD/AslXM2NCqpT4elpwyClWCUjYrRKCRQdImcHdwGOKIxNzpeoE5UQ98+tBoabv9cT OcKev3qJmlRdJoT+nJBfGrBTMrkPBHsLJYmgGPeYb7lKapXCb1FBboT0VwU8vEuedThD pJTTP1bRfJDOhrHovjj8p8Rd/OlVdwUwhSBLDIi4Lntlk7PV9fQjgFElzijyYGr9lYpR miUw== MIME-Version: 1.0 Received: by 10.216.241.202 with SMTP id g52mr12126133wer.212.1346762948049; Tue, 04 Sep 2012 05:49:08 -0700 (PDT) Received: by 10.194.0.145 with HTTP; Tue, 4 Sep 2012 05:49:07 -0700 (PDT) In-Reply-To: <20120903124642.GI33100@deviant.kiev.zoral.com.ua> References: <20120903124642.GI33100@deviant.kiev.zoral.com.ua> Date: Tue, 4 Sep 2012 14:49:07 +0200 Message-ID: From: Svatopluk Kraus To: Konstantin Belousov Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-current@freebsd.org Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 12:49:09 -0000 On Mon, Sep 3, 2012 at 2:46 PM, Konstantin Belousov wrote: > On Mon, Sep 03, 2012 at 12:35:08PM +0200, Svatopluk Kraus wrote: >> Hi, >> >> I found out that while the running excecutables and a dynamic linker >> are protected against writing (ETXTBSY), the loaded shared libraries >> are not protected. The libraries are mapped by mmap() in dynamic >> linker (rtld) and there is no way how to set VV_TEXT flag on the >> libraries vnodes in mmap() code. >> >> In linux compability code \compat\linux\linux_misc.c, linux_uselib() >> sets VV_TEXT flags on a library vnode. In Solaris, MAP_TEXT flag >> exists which informs mmap() that the mapped region will be used >> primarily for executing instructions (for better MMU utilization). >> With these on mind, I propose to implement MAP_TEXT option in mmap() >> and in case that underlying object is a vnode, set VV_TEXT flag on it. >> >> I already have implemented it and with rtld map_object() patch it >> works fine for me (of course). The rtld patch looks easy, however I'm >> not sure about mmap patch. >> >> After some investigation, it looks that VV_TEXT once set on a vnode >> remains set until last reference on the vnode is left. So, I don't >> bother with VV_TEXT unset in munmap() to be consistent. The >> executables and dynamic linker are activated in kernel, so VV_TEXT is >> set before activation and cleared if something failed. Shared library >> activation is done in dynamic linker (i.e., in userland). It's done in >> steps and mmaping the library is one from them. So, I think that >> VV_TEXT can be set in mmap() just after everything is finished >> successfully. > This is right, the object reference counter is also used as > VV_TEXT counter. It is somewhat unaccurate, but in practice does > not cause issues. > >> >> The patch itself is implemented in vm_mmap_vnode(). If I want to set >> VV_TEXT flag on a vnode, I need an exclusive lock. In current code, >> the exclusive lock flag is (mis)used as a flag for >> vnode_pager_update_writecount() call. (I hope that I didn't miss >> something.) So, the patch is bigger slightly. >> >> I defined the MAP_TEXT flag in extented flags sections. However, I'm >> feeling the relation to MAP_STACK flag, but not sure if and when >> reserved flags (in other flags section) can be re-used. >> >> Svata >> >> >> Index: libexec/rtld-elf/map_object.c >> =================================================================== >> --- libexec/rtld-elf/map_object.c (revision 239770) >> +++ libexec/rtld-elf/map_object.c (working copy) >> @@ -199,7 +199,8 @@ >> data_prot = convert_prot(segs[i]->p_flags); >> data_flags = convert_flags(segs[i]->p_flags) | MAP_FIXED; >> if (mmap(data_addr, data_vlimit - data_vaddr, data_prot, >> - data_flags | MAP_PREFAULT_READ, fd, data_offset) == (caddr_t) -1) { >> + data_flags | MAP_PREFAULT_READ | MAP_TEXT, fd, data_offset) == >> + (caddr_t) -1) { > I am not sure that we shall mark all segments mappings with MAP_TEXT. > I understand the logic of the change, since we do not want data segment > to be changed under us. Still, having MAP_TEXT for non-text segments looks > strange. > I agree. However, only way how to recognize a text segment is an executable flag set. The new patch for map_object.c is following: Index: libexec/rtld-elf/map_object.c =================================================================== --- libexec/rtld-elf/map_object.c (revision 239770) +++ libexec/rtld-elf/map_object.c (working copy) @@ -442,5 +442,10 @@ */ if (!(elfflags & PF_W)) flags |= MAP_NOCORE; + /* + * Executable mappings are marked "MAP_TEXT". + */ + if (elfflags & PF_X) + flags |= MAP_TEXT; return flags; } >> _rtld_error("%s: mmap of data failed: %s", path, >> rtld_strerror(errno)); >> goto error1; >> Index: sys/vm/vm_mmap.c >> =================================================================== >> --- sys/vm/vm_mmap.c (revision 239770) >> +++ sys/vm/vm_mmap.c (working copy) >> @@ -1258,10 +1258,13 @@ >> struct mount *mp; >> struct ucred *cred; >> int error, flags, locktype, vfslocked; >> + int writeable_shared; >> >> mp = vp->v_mount; >> cred = td->td_ucred; >> - if ((*maxprotp & VM_PROT_WRITE) && (*flagsp & MAP_SHARED)) >> + flags = *flagsp; >> + writeable_shared = ((*maxprotp & VM_PROT_WRITE) && (flags & MAP_SHARED)); >> + if (writeable_shared || ((flags & MAP_TEXT) != 0)) >> locktype = LK_EXCLUSIVE; >> else >> locktype = LK_SHARED; >> @@ -1271,7 +1274,6 @@ >> return (error); >> } >> foff = *foffp; >> - flags = *flagsp; >> obj = vp->v_object; >> if (vp->v_type == VREG) { >> /* >> @@ -1294,7 +1296,7 @@ >> return (error); >> } >> } >> - if (locktype == LK_EXCLUSIVE) { >> + if (writeable_shared) { >> *writecounted = TRUE; >> vnode_pager_update_writecount(obj, 0, objsize); >> } >> @@ -1337,6 +1339,14 @@ >> error = ENOMEM; >> goto done; >> } >> + /* >> + * If MAP_TEXT is announced, set VV_TEXT so no one can write >> + * to the executable. >> + */ >> + if ((flags & MAP_TEXT) != 0) { >> + ASSERT_VOP_ELOCKED(vp, "vv_text"); >> + vp->v_vflag |= VV_TEXT; >> + } > I do not think we want to set VV_TEXT for device vnodes. > I agree too. However, my patch doesn't set VV_TEXT for device vnodes. Device vnodes never enter into patched part of code. Svata From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 13:00:45 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2212F106564A for ; Tue, 4 Sep 2012 13:00:45 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 7B6BC8FC12 for ; Tue, 4 Sep 2012 13:00:44 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q84D0pqA064967; Tue, 4 Sep 2012 16:00:51 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q84D0deI044748; Tue, 4 Sep 2012 16:00:39 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q84D0d8x044747; Tue, 4 Sep 2012 16:00:39 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 4 Sep 2012 16:00:39 +0300 From: Konstantin Belousov To: Svatopluk Kraus Message-ID: <20120904130039.GX33100@deviant.kiev.zoral.com.ua> References: <20120903124642.GI33100@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="a8GRLKP4M7ovxuH3" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-current@freebsd.org Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 13:00:45 -0000 --a8GRLKP4M7ovxuH3 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Sep 04, 2012 at 02:49:07PM +0200, Svatopluk Kraus wrote: > On Mon, Sep 3, 2012 at 2:46 PM, Konstantin Belousov = wrote: > > On Mon, Sep 03, 2012 at 12:35:08PM +0200, Svatopluk Kraus wrote: > >> Hi, > >> > >> I found out that while the running excecutables and a dynamic linker > >> are protected against writing (ETXTBSY), the loaded shared libraries > >> are not protected. The libraries are mapped by mmap() in dynamic > >> linker (rtld) and there is no way how to set VV_TEXT flag on the > >> libraries vnodes in mmap() code. > >> > >> In linux compability code \compat\linux\linux_misc.c, linux_uselib() > >> sets VV_TEXT flags on a library vnode. In Solaris, MAP_TEXT flag > >> exists which informs mmap() that the mapped region will be used > >> primarily for executing instructions (for better MMU utilization). > >> With these on mind, I propose to implement MAP_TEXT option in mmap() > >> and in case that underlying object is a vnode, set VV_TEXT flag on it. > >> > >> I already have implemented it and with rtld map_object() patch it > >> works fine for me (of course). The rtld patch looks easy, however I'm > >> not sure about mmap patch. > >> > >> After some investigation, it looks that VV_TEXT once set on a vnode > >> remains set until last reference on the vnode is left. So, I don't > >> bother with VV_TEXT unset in munmap() to be consistent. The > >> executables and dynamic linker are activated in kernel, so VV_TEXT is > >> set before activation and cleared if something failed. Shared library > >> activation is done in dynamic linker (i.e., in userland). It's done in > >> steps and mmaping the library is one from them. So, I think that > >> VV_TEXT can be set in mmap() just after everything is finished > >> successfully. > > This is right, the object reference counter is also used as > > VV_TEXT counter. It is somewhat unaccurate, but in practice does > > not cause issues. > > > >> > >> The patch itself is implemented in vm_mmap_vnode(). If I want to set > >> VV_TEXT flag on a vnode, I need an exclusive lock. In current code, > >> the exclusive lock flag is (mis)used as a flag for > >> vnode_pager_update_writecount() call. (I hope that I didn't miss > >> something.) So, the patch is bigger slightly. > >> > >> I defined the MAP_TEXT flag in extented flags sections. However, I'm > >> feeling the relation to MAP_STACK flag, but not sure if and when > >> reserved flags (in other flags section) can be re-used. > >> > >> Svata > >> > >> > >> Index: libexec/rtld-elf/map_object.c > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > >> --- libexec/rtld-elf/map_object.c (revision 239770) > >> +++ libexec/rtld-elf/map_object.c (working copy) > >> @@ -199,7 +199,8 @@ > >> data_prot =3D convert_prot(segs[i]->p_flags); > >> data_flags =3D convert_flags(segs[i]->p_flags) | MAP_FIXED; > >> if (mmap(data_addr, data_vlimit - data_vaddr, data_prot, > >> - data_flags | MAP_PREFAULT_READ, fd, data_offset) =3D=3D (caddr= _t) -1) { > >> + data_flags | MAP_PREFAULT_READ | MAP_TEXT, fd, data_offset) = =3D=3D > >> + (caddr_t) -1) { > > I am not sure that we shall mark all segments mappings with MAP_TEXT. > > I understand the logic of the change, since we do not want data segment > > to be changed under us. Still, having MAP_TEXT for non-text segments lo= oks > > strange. > > >=20 > I agree. However, only way how to recognize a text segment is an > executable flag set. The new patch for map_object.c is following: >=20 > Index: libexec/rtld-elf/map_object.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- libexec/rtld-elf/map_object.c (revision 239770) > +++ libexec/rtld-elf/map_object.c (working copy) > @@ -442,5 +442,10 @@ > */ > if (!(elfflags & PF_W)) > flags |=3D MAP_NOCORE; > + /* > + * Executable mappings are marked "MAP_TEXT". > + */ > + if (elfflags & PF_X) > + flags |=3D MAP_TEXT; > return flags; > } >=20 >=20 > >> _rtld_error("%s: mmap of data failed: %s", path, > >> rtld_strerror(errno)); > >> goto error1; > >> Index: sys/vm/vm_mmap.c > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > >> --- sys/vm/vm_mmap.c (revision 239770) > >> +++ sys/vm/vm_mmap.c (working copy) > >> @@ -1258,10 +1258,13 @@ > >> struct mount *mp; > >> struct ucred *cred; > >> int error, flags, locktype, vfslocked; > >> + int writeable_shared; > >> > >> mp =3D vp->v_mount; > >> cred =3D td->td_ucred; > >> - if ((*maxprotp & VM_PROT_WRITE) && (*flagsp & MAP_SHARED)) > >> + flags =3D *flagsp; > >> + writeable_shared =3D ((*maxprotp & VM_PROT_WRITE) && (flags & MA= P_SHARED)); > >> + if (writeable_shared || ((flags & MAP_TEXT) !=3D 0)) > >> locktype =3D LK_EXCLUSIVE; > >> else > >> locktype =3D LK_SHARED; > >> @@ -1271,7 +1274,6 @@ > >> return (error); > >> } > >> foff =3D *foffp; > >> - flags =3D *flagsp; > >> obj =3D vp->v_object; > >> if (vp->v_type =3D=3D VREG) { > >> /* > >> @@ -1294,7 +1296,7 @@ > >> return (error); > >> } > >> } > >> - if (locktype =3D=3D LK_EXCLUSIVE) { > >> + if (writeable_shared) { > >> *writecounted =3D TRUE; > >> vnode_pager_update_writecount(obj, 0, objsize); > >> } > >> @@ -1337,6 +1339,14 @@ > >> error =3D ENOMEM; > >> goto done; > >> } > >> + /* > >> + * If MAP_TEXT is announced, set VV_TEXT so no one can write > >> + * to the executable. > >> + */ > >> + if ((flags & MAP_TEXT) !=3D 0) { > >> + ASSERT_VOP_ELOCKED(vp, "vv_text"); > >> + vp->v_vflag |=3D VV_TEXT; > >> + } > > I do not think we want to set VV_TEXT for device vnodes. > > >=20 > I agree too. However, my patch doesn't set VV_TEXT for device vnodes. > Device vnodes never enter into patched part of code. Hm, yes. Anyway, after thinking about the patch more, I see two issues: 1. You are setting VV_TEXT without checking v_writecount. This basically nullifies the main reason for the patch, since existing writer can still write or truncate the shared library after the mapping. 2. I do not see what would prevent malicious local user from mmaping arbitrary file readonly with MAP_TEXT, thus blocking any modifications to the file. Note that this is not a problem for executables, because kernel only sets VV_TEXT on executables if +x permission is set and file is valid binary which kernel is able to execute. E.g. you might block log writes with VV_TEXT, or other user editing session or whatever, having just read access to corresponding files. Am I wrong ? --a8GRLKP4M7ovxuH3 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlBF+3cACgkQC3+MBN1Mb4jnpwCg390Q8EZRkkl92IXUORPYBD0h HPMAoM5sBZeFj3Ao5m9hLDKYypVsg43G =dcFT -----END PGP SIGNATURE----- --a8GRLKP4M7ovxuH3-- From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 13:09:50 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7A5F910657B7; Tue, 4 Sep 2012 13:09:50 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from onyx.glenbarber.us (onyx.glenbarber.us [IPv6:2607:fc50:1000:c200::face]) by mx1.freebsd.org (Postfix) with ESMTP id 0C1D68FC0A; Tue, 4 Sep 2012 13:09:50 +0000 (UTC) Received: from glenbarber.us (75-146-225-65-Philadelphia.hfc.comcastbusiness.net [75.146.225.65]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: gjb) by onyx.glenbarber.us (Postfix) with ESMTPSA id 5335D23F645; Tue, 4 Sep 2012 09:09:49 -0400 (EDT) Date: Tue, 4 Sep 2012 09:09:47 -0400 From: Glen Barber To: AN Message-ID: <20120904130947.GA1452@glenbarber.us> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="82I3+IH0IqGh5yIs" Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-current@freebsd.org Subject: Re: recent update seems to break portupgrade X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 13:09:50 -0000 --82I3+IH0IqGh5yIs Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Sep 04, 2012 at 03:14:42AM -0400, AN wrote: > FreeBSD FBSD10 10.0-CURRENT FreeBSD 10.0-CURRENT #18 r240078: Mon Sep 3= =20 > 17:41:46 EDT 2012 root@FBSD10:/usr/obj/usr/src/sys/MYKERNEL amd64 >=20 > # cat /etc/make.conf > OVERRIDE_LINUX_BASE_PORT=3Df10 > QT4_OPTIONS=3D QGTKSTYLE >=20 > # added by use.perl 2012-04-04 01:11:13 > PERL_VERSION=3D5.14.2 > MALLOC_PRODUCTION=3Dyes > WITH_PKGNG=3Dno >=20 I think this should be 'WITHOUT_PKGNG=3Dyes', since 'WITH_PKGNG=3Dno' still sets WITH_PKGNG. Glen --82I3+IH0IqGh5yIs Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBCAAGBQJQRf2bAAoJEFJPDDeguUajIfYH/RLggkB12C+bs+OIWCnGUwVi lkUyGb/9ihXlc7SrZOAnucH14eKd69l8WKOO6MXaVZiZLcK7pWCmZwbbi+2hJ/A6 09hm/xEi+qM46lh1G7OlSJJ8yVJnMcPAJ0czDN2RZwiIJBS+LkrkS0iGR/wAoCpP mOHFjFg55B70VwTF9fsICejq5jNDXPhGNGWzLY5rQdmb9603WExbLW6ve76batrm w/ONtfV89FY9JQxjLIoklDBntd4FhJDxnUkiB+V4GsrCHm2QUE2RX0oOTJ8AxK1E DnqP8Ipso4JZbD2Cb9nedIXfqmnYOQEdwAFDTAM/o2sblqgifpCHGc+EIBN4ZNk= =9eLL -----END PGP SIGNATURE----- --82I3+IH0IqGh5yIs-- From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 13:33:30 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 433C8106564A; Tue, 4 Sep 2012 13:33:30 +0000 (UTC) (envelope-from davide.italiano@gmail.com) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id CD5418FC14; Tue, 4 Sep 2012 13:33:29 +0000 (UTC) Received: by vbmv11 with SMTP id v11so8642058vbm.13 for ; Tue, 04 Sep 2012 06:33:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=ox0rklEFi1TX/qj2x59HK4+CLASme1LIukTxR9jlBFM=; b=uFe5UFrxdn4AzOnWes2eZEC9czrrgjR3HDN8bORCQamdwyZzlxqyYz1dyOSF2SOkMn Cs7VJF1Kh4YlLbW1agyOYk2a+hlx1Neiue2ywmwxDe7iSuhSu7ZIc3KoYwgC5XCza/vj VLzkabd3+02fu4IqND306MSLTZbf3Tfg3Yxy7SUbPIDV8PY5J06sJrN1QqMhIVTdulZo 2ZeCJhUa4NiR+VEvkAHXRt5sKlTpeyCp73APEkQtVkoWfTiWWqTSKPwO+T6VCg427Mhr v93KEaAFbf2FuUcrc+BQsG1MYAlTqSDOCyHqdA2PpMpPMpH6vNCki6OhT4yF+OrrsO0g bPpA== MIME-Version: 1.0 Received: by 10.52.66.2 with SMTP id b2mr12068440vdt.92.1346765603305; Tue, 04 Sep 2012 06:33:23 -0700 (PDT) Sender: davide.italiano@gmail.com Received: by 10.58.55.230 with HTTP; Tue, 4 Sep 2012 06:33:23 -0700 (PDT) In-Reply-To: <60A32BCE-756A-4A0D-A24D-7AC42F71BD1A@netasq.com> References: <0D822C75-7240-42E3-8780-18CABC9C45A1@netasq.com> <20120903212340.GE88081@ithaqua.etoilebsd.net> <60A32BCE-756A-4A0D-A24D-7AC42F71BD1A@netasq.com> Date: Tue, 4 Sep 2012 15:33:23 +0200 X-Google-Sender-Auth: JwtU-UnYFOsIHLFmc9JVlXo_6Xk Message-ID: From: Davide Italiano To: Fabien Thomas Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Baptiste Daroussin , freebsd-current Subject: Re: [CFT] hwpmc support for Intel Ivy Bridge X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 13:33:30 -0000 On Tue, Sep 4, 2012 at 9:35 AM, Fabien Thomas wr= ote: > > Le 3 sept. 2012 =E0 23:23, Baptiste Daroussin a =E9crit : > >> On Mon, Sep 03, 2012 at 02:04:21PM +0200, Fabien Thomas wrote: >>> Hi, >>> >>> Find a patch that add Intel Ivy Bridge support to hwpmc(9). >>> The patch also support offcore RSP token for Sandy Bridge. >>> Note: No uncore support. >>> >>> Tested on: >>> Intel(R) Xeon(R) CPU E3-1265L V2 @ 2.50GHz (2494.35-MHz K8-class CPU) >>> Origin =3D "GenuineIntel" Id =3D 0x306a9 Family =3D 6 Model =3D 3a = Stepping =3D 9 >>> >>> http://people.freebsd.org/~fabient/patch-hwpmc_ivy_bridge_head >>> >>> cd /usr/src && patch -p0 < patch-hwpmc_ivy_bridge_head >>> and rebuild world. >>> >>> >>> Fabien >> >> World building, I'll keep you in touch >> >> Is there any particulat testing, I can do other than using the new world= ? > > Thanks for testing! > > Yes you can test it quickly using top mode for example: > > kldload hwpmc > pmccontrol -L to list all events > > some test: > > - offcore > pmcstat -SOFF_CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY -w1 -T > > - unhalted cycle (programmable counter) > pmcstat -n2000000 -SCPU_CLK_UNHALTED.THREAD_P -w4 -T > > - INSTR_RETIRED_ANY (fixed counter) > pmcstat -n2000000 -S INSTR_RETIRED_ANY -w4 -T > > - Soft counter: > pmcstat -SPAGE_FAULT.ALL -w4 -T > > all at the same time (and change view by pressing 'p': > pmcstat -SOFF_CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY -n2000000 -S= CPU_CLK_UNHALTED.THREAD_P -n2000000 -S INSTR_RETIRED_ANY -SPAGE_FAULT.ALL= -w1 -T > > > all events are described in the man page pmc.ivybridge > > >> >> regards, >> Bapt > I suggest you to run the script you can find in /head/tools/test/hwpmc/pmctest.py , courtesy of gnn@. Thanks Davide From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 15:32:05 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BF8F1106564A for ; Tue, 4 Sep 2012 15:32:05 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 74B738FC14 for ; Tue, 4 Sep 2012 15:32:05 +0000 (UTC) Received: by yenl7 with SMTP id l7so1248864yen.13 for ; Tue, 04 Sep 2012 08:32:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=qj2f9TirLFFHgURfkywCwB1geFPoQWdauxiWFQpAWWs=; b=XqJnUJ+mO2XD9puuBiurQ5yxPTaRxrtFWx2hOZc1zKqw/tGt/jsFRl03xDOtzbotXK QQnFRxXKmPjzfO7n8vyshCtGvSnRRMa0yeprLLjCg3UyVLjcsbGFWpUj0lUMtjpwaSrK 0ESoZYwVq5Vxcu6GZrOj58SKvXxRE3bSEBmhBX2Lv8hIbGz0KuTKMUykICwf7abfDB69 DnySZZ9lANci5aLFeAftUVlzJIfOVw/shgF2QKNgw/P7sY7Sv5qnMNdLfP1aQ7Wu8kD1 doPpw4Ge5IpD2HDrZH8a3r0ln+pvbIwSQVTPPTzdwf8Cm4c1MzDi3ofbyz6WCAPVbWl4 fG1w== Received: by 10.236.183.168 with SMTP id q28mr18227380yhm.45.1346772725006; Tue, 04 Sep 2012 08:32:05 -0700 (PDT) Received: from [10.30.101.53] ([209.117.142.2]) by mx.google.com with ESMTPS id t57sm30394580yhg.0.2012.09.04.08.32.02 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 04 Sep 2012 08:32:04 -0700 (PDT) Sender: Warner Losh Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <50417ADC.3030408@FreeBSD.org> Date: Tue, 4 Sep 2012 09:31:57 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <50417ADC.3030408@FreeBSD.org> To: John Baldwin X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQmIbKAQV07IoIQDVtCsyWDr71qRZ7dbIJ0XeYwOwAjxV9E9KmhXk+DUzFLYM48NeGjj5Sfd Cc: current@FreeBSD.org Subject: Re: [PATCH] Add locking to ahb(4) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 15:32:05 -0000 I have a full EISA system and a dozen of these cards, and other EISA = cards, if anybody wants to take up tilting at that windmill. I'm in = Denver CO, and will drive it to your house, within reason. The system = is a dual CPU 533MHz Digital system with all the config disks, etc, you = need to support changing EISA cards. It ran FreeBSD 5.x and 6.x pretty = well. Sadly, my other commitments keep me from testing this myself... Warner On Aug 31, 2012, at 9:02 PM, John Baldwin wrote: > I have patches to add locking to ahb(4) and mark it MPSAFE. The > patches are from HEAD but should apply to 8 or 9. If you test it on 8 > or 9 please enable INVARIANTS for at least the initial testing. = Thanks. >=20 > http://www.FreeBSD.org/~jhb/patches/ahb_locking.patch >=20 > --=20 > John Baldwin > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to = "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 16:01:10 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3926D1065678 for ; Tue, 4 Sep 2012 16:01:10 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id E85B18FC1B for ; Tue, 4 Sep 2012 16:01:09 +0000 (UTC) Received: by vbmv11 with SMTP id v11so8913428vbm.13 for ; Tue, 04 Sep 2012 09:01:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=DnOHLtydX+Sh61pGxXOdmBlytLIX5UTk5x8JnnelkzU=; b=O4WiAgcYhRP28v+kN3RTCJkaepD+eC8ef7wni7XMaTqYJJxMNmkRsc3ZBNRGxDqboG aD+xCy2TOdwg+vHZvxNnC7ZFdCg8m7FJ4tUR0X5KYjHxYV+PNjgRg/B8pqrloFVBVaEO ryxA1qBX6aI2si9lbvmAEjr8QqB+dwXEXbnq7yAWoQLVrHgtGUO3eYdPeYVPnEj/7XDt njt8uGY41r3dThBNsVtTUf1UQlM1HeIOxdw0ljpE+THaGEqJR6epQ1SwUl3PkqQ4tbbX b19hhgDruGK0XQR3zhtmU8+jgf5NtGyuWibYCAmCQe6wxH6fEIfT2twolmoPHALkv3h2 Q2FQ== MIME-Version: 1.0 Received: by 10.52.27.239 with SMTP id w15mr9877000vdg.96.1346774469053; Tue, 04 Sep 2012 09:01:09 -0700 (PDT) Received: by 10.58.207.114 with HTTP; Tue, 4 Sep 2012 09:01:09 -0700 (PDT) Date: Tue, 4 Sep 2012 12:01:09 -0400 Message-ID: From: Ryan Stone To: FreeBSD Current Content-Type: text/plain; charset=ISO-8859-1 Subject: machdep.hyperthreading_allowed does not affect SMT cores X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 16:01:10 -0000 I have a Intel Sandy Bridge system that reports that it has SMT cores instead of HTT(under a derivative of FreeBSD 8.2). I'll admit that I don't at all understand the distinction between the two -- I thought that HTT was just Intel's name for SMT. In any case, is there any reason that machdep.hyperthreading_allowed should not apply to SMT cores, too? From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 16:06:50 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0EE141065716 for ; Tue, 4 Sep 2012 16:06:50 +0000 (UTC) (envelope-from bryan@shatow.net) Received: from secure.xzibition.com (secure.xzibition.com [173.160.118.92]) by mx1.freebsd.org (Postfix) with ESMTP id A64568FC19 for ; Tue, 4 Sep 2012 16:06:49 +0000 (UTC) DomainKey-Signature: a=rsa-sha1; c=nofws; d=shatow.net; h=message-id :date:from:mime-version:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; q=dns; s=sweb; b=TQgKLQ +Cb/fStmlEmHz8Ovw+AQRT4BJ0VKGBBzCSTPQ7uXcaHuhaK09TgVFrqFcCEUxjNm hor3G1Tuvc4KOq0XnWnRT+/S4QkcP6EwYaRJCja3dSu+kZHrU8WB4PSdOJZEqYe0 9/4/yNUHJOERUWggR/+qqSgkIwH3W4bmZtqQU= DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=shatow.net; h=message-id :date:from:mime-version:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; s=sweb; bh=V71J5p/CcMWr ufvKhfgUxaEvEJ0U34vzZ2gy47XazAE=; b=VOA7qZXgW7vfl7RaBG4QUpokReEo Eo725/rmxgKjj71UOXaZvTEBo7+4NgznMgDHTxcdOpxyrop+OmHEhF9px666vIq/ u4mB2FrZTjhyqJQYRN9mmyfit4XoM2XuCblNu7d7s8Gi0fPNLtt0OQGlsHei3bv9 Zq/Rd51WZatYg0Y= Received: (qmail 61098 invoked from network); 4 Sep 2012 11:06:47 -0500 Received: from unknown (HELO ?192.168.0.74?) (bryan@shatow.net@74.94.87.209) by sweb.xzibition.com with ESMTPA; 4 Sep 2012 11:06:47 -0500 Message-ID: <50462718.1030406@shatow.net> Date: Tue, 04 Sep 2012 11:06:48 -0500 From: Bryan Drewery User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 References: <50457942.90909@shatow.net> In-Reply-To: <50457942.90909@shatow.net> X-Enigmail-Version: 1.4.4 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, Ports FreeBSD Subject: Re: recent update seems to break portupgrade X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 16:06:50 -0000 On 9/3/2012 10:45 PM, Bryan Drewery wrote: > These errors are a problem with bsd.port.mk. I've been meaning to write > up a patch for a while now and get to bapt for testing. If he doesn't > get to it first, I'll get one submitted soon. I've filed ports/171326 to address. The warnings are harmless. Bryan From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 19:12:58 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 03F731065686 for ; Tue, 4 Sep 2012 19:12:58 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id BA8558FC19 for ; Tue, 4 Sep 2012 19:12:57 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 2B61CB96E; Tue, 4 Sep 2012 15:12:57 -0400 (EDT) From: John Baldwin To: freebsd-current@freebsd.org Date: Tue, 4 Sep 2012 12:00:27 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) References: <20120904130039.GX33100@deviant.kiev.zoral.com.ua> In-Reply-To: <20120904130039.GX33100@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201209041200.27100.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 04 Sep 2012 15:12:57 -0400 (EDT) Cc: Konstantin Belousov , Svatopluk Kraus Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 19:12:58 -0000 On Tuesday, September 04, 2012 9:00:39 am Konstantin Belousov wrote: > On Tue, Sep 04, 2012 at 02:49:07PM +0200, Svatopluk Kraus wrote: > > On Mon, Sep 3, 2012 at 2:46 PM, Konstantin Belousov wrote: > > > On Mon, Sep 03, 2012 at 12:35:08PM +0200, Svatopluk Kraus wrote: > > >> Hi, > > >> > > >> I found out that while the running excecutables and a dynamic linker > > >> are protected against writing (ETXTBSY), the loaded shared libraries > > >> are not protected. The libraries are mapped by mmap() in dynamic > > >> linker (rtld) and there is no way how to set VV_TEXT flag on the > > >> libraries vnodes in mmap() code. > > >> > > >> In linux compability code \compat\linux\linux_misc.c, linux_uselib() > > >> sets VV_TEXT flags on a library vnode. In Solaris, MAP_TEXT flag > > >> exists which informs mmap() that the mapped region will be used > > >> primarily for executing instructions (for better MMU utilization). > > >> With these on mind, I propose to implement MAP_TEXT option in mmap() > > >> and in case that underlying object is a vnode, set VV_TEXT flag on it. > > >> > > >> I already have implemented it and with rtld map_object() patch it > > >> works fine for me (of course). The rtld patch looks easy, however I'm > > >> not sure about mmap patch. > > >> > > >> After some investigation, it looks that VV_TEXT once set on a vnode > > >> remains set until last reference on the vnode is left. So, I don't > > >> bother with VV_TEXT unset in munmap() to be consistent. The > > >> executables and dynamic linker are activated in kernel, so VV_TEXT is > > >> set before activation and cleared if something failed. Shared library > > >> activation is done in dynamic linker (i.e., in userland). It's done in > > >> steps and mmaping the library is one from them. So, I think that > > >> VV_TEXT can be set in mmap() just after everything is finished > > >> successfully. > > > This is right, the object reference counter is also used as > > > VV_TEXT counter. It is somewhat unaccurate, but in practice does > > > not cause issues. > > > > > >> > > >> The patch itself is implemented in vm_mmap_vnode(). If I want to set > > >> VV_TEXT flag on a vnode, I need an exclusive lock. In current code, > > >> the exclusive lock flag is (mis)used as a flag for > > >> vnode_pager_update_writecount() call. (I hope that I didn't miss > > >> something.) So, the patch is bigger slightly. > > >> > > >> I defined the MAP_TEXT flag in extented flags sections. However, I'm > > >> feeling the relation to MAP_STACK flag, but not sure if and when > > >> reserved flags (in other flags section) can be re-used. > > >> > > >> Svata > > >> > > >> > > >> Index: libexec/rtld-elf/map_object.c > > >> =================================================================== > > >> --- libexec/rtld-elf/map_object.c (revision 239770) > > >> +++ libexec/rtld-elf/map_object.c (working copy) > > >> @@ -199,7 +199,8 @@ > > >> data_prot = convert_prot(segs[i]->p_flags); > > >> data_flags = convert_flags(segs[i]->p_flags) | MAP_FIXED; > > >> if (mmap(data_addr, data_vlimit - data_vaddr, data_prot, > > >> - data_flags | MAP_PREFAULT_READ, fd, data_offset) == (caddr_t) -1) { > > >> + data_flags | MAP_PREFAULT_READ | MAP_TEXT, fd, data_offset) == > > >> + (caddr_t) -1) { > > > I am not sure that we shall mark all segments mappings with MAP_TEXT. > > > I understand the logic of the change, since we do not want data segment > > > to be changed under us. Still, having MAP_TEXT for non-text segments looks > > > strange. > > > > > > > I agree. However, only way how to recognize a text segment is an > > executable flag set. The new patch for map_object.c is following: > > > > Index: libexec/rtld-elf/map_object.c > > =================================================================== > > --- libexec/rtld-elf/map_object.c (revision 239770) > > +++ libexec/rtld-elf/map_object.c (working copy) > > @@ -442,5 +442,10 @@ > > */ > > if (!(elfflags & PF_W)) > > flags |= MAP_NOCORE; > > + /* > > + * Executable mappings are marked "MAP_TEXT". > > + */ > > + if (elfflags & PF_X) > > + flags |= MAP_TEXT; > > return flags; > > } > > > > > > >> _rtld_error("%s: mmap of data failed: %s", path, > > >> rtld_strerror(errno)); > > >> goto error1; > > >> Index: sys/vm/vm_mmap.c > > >> =================================================================== > > >> --- sys/vm/vm_mmap.c (revision 239770) > > >> +++ sys/vm/vm_mmap.c (working copy) > > >> @@ -1258,10 +1258,13 @@ > > >> struct mount *mp; > > >> struct ucred *cred; > > >> int error, flags, locktype, vfslocked; > > >> + int writeable_shared; > > >> > > >> mp = vp->v_mount; > > >> cred = td->td_ucred; > > >> - if ((*maxprotp & VM_PROT_WRITE) && (*flagsp & MAP_SHARED)) > > >> + flags = *flagsp; > > >> + writeable_shared = ((*maxprotp & VM_PROT_WRITE) && (flags & MAP_SHARED)); > > >> + if (writeable_shared || ((flags & MAP_TEXT) != 0)) > > >> locktype = LK_EXCLUSIVE; > > >> else > > >> locktype = LK_SHARED; > > >> @@ -1271,7 +1274,6 @@ > > >> return (error); > > >> } > > >> foff = *foffp; > > >> - flags = *flagsp; > > >> obj = vp->v_object; > > >> if (vp->v_type == VREG) { > > >> /* > > >> @@ -1294,7 +1296,7 @@ > > >> return (error); > > >> } > > >> } > > >> - if (locktype == LK_EXCLUSIVE) { > > >> + if (writeable_shared) { > > >> *writecounted = TRUE; > > >> vnode_pager_update_writecount(obj, 0, objsize); > > >> } > > >> @@ -1337,6 +1339,14 @@ > > >> error = ENOMEM; > > >> goto done; > > >> } > > >> + /* > > >> + * If MAP_TEXT is announced, set VV_TEXT so no one can write > > >> + * to the executable. > > >> + */ > > >> + if ((flags & MAP_TEXT) != 0) { > > >> + ASSERT_VOP_ELOCKED(vp, "vv_text"); > > >> + vp->v_vflag |= VV_TEXT; > > >> + } > > > I do not think we want to set VV_TEXT for device vnodes. > > > > > > > I agree too. However, my patch doesn't set VV_TEXT for device vnodes. > > Device vnodes never enter into patched part of code. > Hm, yes. > > Anyway, after thinking about the patch more, I see two issues: > > 1. You are setting VV_TEXT without checking v_writecount. This basically > nullifies the main reason for the patch, since existing writer can still > write or truncate the shared library after the mapping. > > 2. I do not see what would prevent malicious local user from mmaping > arbitrary file readonly with MAP_TEXT, thus blocking any modifications > to the file. Note that this is not a problem for executables, because > kernel only sets VV_TEXT on executables if +x permission is set and > file is valid binary which kernel is able to execute. > > E.g. you might block log writes with VV_TEXT, or other user editing > session or whatever, having just read access to corresponding files. > > Am I wrong ? Hmm, I do think 2) is a bit of a show-stopper. I do wonder why one needs MAP_TEXT at all or if you could key this off of mmap() with PROT_EXEC? Do we require +x permissions for PROT_EXEC? No, it seems we only require a file opened with FREAD. Hmm, perhaps rtld could open a separate fd for PROT_EXEC mappings that used O_EXEC and mmap()'ing an O_EXEC fd could enable VV_TEXT? That would require a file to have +x permisson for an mmap() to enable VV_TEXT. It would also make MAP_TEXT unneeded. -- John Baldwin From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 19:23:24 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6F420106564A for ; Tue, 4 Sep 2012 19:23:24 +0000 (UTC) (envelope-from bapt@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 512078FC20; Tue, 4 Sep 2012 19:23:24 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q84JNONT056013; Tue, 4 Sep 2012 19:23:24 GMT (envelope-from bapt@FreeBSD.org) Received: (from bapt@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q84JNOwP055964; Tue, 4 Sep 2012 19:23:24 GMT (envelope-from bapt@FreeBSD.org) X-Authentication-Warning: freefall.freebsd.org: bapt set sender to bapt@FreeBSD.org using -f Date: Tue, 4 Sep 2012 21:23:21 +0200 From: Baptiste Daroussin To: Fabien Thomas Message-ID: <20120904192320.GF88081@ithaqua.etoilebsd.net> References: <0D822C75-7240-42E3-8780-18CABC9C45A1@netasq.com> <20120903212340.GE88081@ithaqua.etoilebsd.net> <60A32BCE-756A-4A0D-A24D-7AC42F71BD1A@netasq.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="iBwuxWUsK/REspAd" Content-Disposition: inline In-Reply-To: <60A32BCE-756A-4A0D-A24D-7AC42F71BD1A@netasq.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-current Subject: Re: [CFT] hwpmc support for Intel Ivy Bridge X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 19:23:24 -0000 --iBwuxWUsK/REspAd Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Sep 04, 2012 at 09:35:07AM +0200, Fabien Thomas wrote: >=20 > Le 3 sept. 2012 =E0 23:23, Baptiste Daroussin a =E9crit : >=20 > > On Mon, Sep 03, 2012 at 02:04:21PM +0200, Fabien Thomas wrote: > >> Hi, > >>=20 > >> Find a patch that add Intel Ivy Bridge support to hwpmc(9). > >> The patch also support offcore RSP token for Sandy Bridge. > >> Note: No uncore support. > >>=20 > >> Tested on: > >> Intel(R) Xeon(R) CPU E3-1265L V2 @ 2.50GHz (2494.35-MHz K8-class CPU) > >> Origin =3D "GenuineIntel" Id =3D 0x306a9 Family =3D 6 Model =3D 3a = Stepping =3D 9 > >>=20 > >> http://people.freebsd.org/~fabient/patch-hwpmc_ivy_bridge_head > >>=20 > >> cd /usr/src && patch -p0 < patch-hwpmc_ivy_bridge_head=20 > >> and rebuild world. > >>=20 > >>=20 > >> Fabien > >=20 > > World building, I'll keep you in touch > >=20 > > Is there any particulat testing, I can do other than using the new worl= d? >=20 > Thanks for testing! >=20 > Yes you can test it quickly using top mode for example: >=20 > kldload hwpmc > pmccontrol -L to list all events >=20 > some test: >=20 > - offcore > pmcstat -SOFF_CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY -w1 -T >=20 > - unhalted cycle (programmable counter) > pmcstat -n2000000 -SCPU_CLK_UNHALTED.THREAD_P -w4 -T >=20 > - INSTR_RETIRED_ANY (fixed counter) > pmcstat -n2000000 -S INSTR_RETIRED_ANY -w4 -T >=20 > - Soft counter: > pmcstat -SPAGE_FAULT.ALL -w4 -T >=20 > all at the same time (and change view by pressing 'p': > pmcstat -SOFF_CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY -n2000000 -S= CPU_CLK_UNHALTED.THREAD_P -n2000000 -S INSTR_RETIRED_ANY -SPAGE_FAULT.ALL= -w1 -T >=20 >=20 > all events are described in the man page pmc.ivybridge >=20 >=20 > >=20 > > regards, > > Bapt >=20 Hi, here are the results # pmccontrol -L SOFT CLOCK.STAT CLOCK.HARD LOCK.FAILED PAGE_FAULT.WRITE PAGE_FAULT.READ PAGE_FAULT.ALL # pmcstat -SOFF_CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY -w1 -T pmcstat: ERROR: Cannot allocate system-mode pmc with specification "OFF_COR= E_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY": Invalid argument # pmcstat -n2000000 -S INSTR_RETIRED_ANY -w4 -T pmcstat: ERROR: Cannot allocate system-mode pmc with specification "INSTR_R= ETIRED_ANY": Invalid argument # pmcstat -n2000000 -S INSTR_RETIRED_ANY -w4 -T pmcstat: ERROR: Cannot allocate system-mode pmc with specification "INSTR_R= ETIRED_ANY": Invalid argument # pmcstat -SPAGE_FAULT.ALL -w4 -T (This one seems to work correctly # pmcstat -SOFF_CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY -n2000000 -S= CPU_CLK_UNHALTED.THREAD_P -n2000000 -S INSTR_RETIRED_ANY -SPAGE_FAULT pmcstat: ERROR: Cannot allocate system-mode pmc with specification "OFF_COR= E_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY": Invalid argument The pmctest.py fails:=20 # ./test/hwpmc/pmctest.py Traceback (most recent call last): File "./test/hwpmc/pmctest.py", line 94, in main() File "./test/hwpmc/pmctest.py", line 81, in main stdout=3DPIPE) File "/usr/local/lib/python2.7/subprocess.py", line 679, in __init__ errread, errwrite) File "/usr/local/lib/python2.7/subprocess.py", line 1249, in _execute_chi= ld raise child_exception TypeError: execv() arg 2 must contain only strings regards, Bapt --iBwuxWUsK/REspAd Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlBGVSgACgkQ8kTtMUmk6EzehgCfd3fpYjeqp+VPJIAud9wJfY6G xDoAn2XuY0uCVHEYo+myoJ3E9p52Zk53 =QOo+ -----END PGP SIGNATURE----- --iBwuxWUsK/REspAd-- From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 20:02:07 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BFED8106566C; Tue, 4 Sep 2012 20:02:07 +0000 (UTC) (envelope-from davide.italiano@gmail.com) Received: from mail-gg0-f182.google.com (mail-gg0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 63A388FC14; Tue, 4 Sep 2012 20:02:07 +0000 (UTC) Received: by ggnk4 with SMTP id k4so1448244ggn.13 for ; Tue, 04 Sep 2012 13:02:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=jCgkL5oc2o9zXEfH+PG+TbRmqlOVivW8NxCsfClZirc=; b=sFBTSs4pIr35DkvPlAoYD4U+hvF6ytjZWSYLfDtMFZwMLCrfVZPMxd4WhCLJi8vc/C ndU37XX6KkeDTnlpMJS1ELb1sqnA6V0yUHvPQw5qpcXyM4SYsup1vTDSm6+M9P13eBI7 Xs3byazCrYNrwcCkXoz+4w3RYJdLsFFENqD5alDvpzepD62CkGc+8eHV4pIWDZFKBn8M ZNwzZomJu9iNQ6W89IpnfcZODVBlpq81oqscqrJ0zT7jN/GC8tvCorT40APrXWl8pl57 D3ZzlMX6ry5HpOb5FJsJY0R2w44gFf8Bt+KaqDALsbPtRfANblGTNF8qnroPCeYtQb1y b9yQ== MIME-Version: 1.0 Received: by 10.58.74.3 with SMTP id p3mr3176vev.27.1346788926546; Tue, 04 Sep 2012 13:02:06 -0700 (PDT) Sender: davide.italiano@gmail.com Received: by 10.58.55.230 with HTTP; Tue, 4 Sep 2012 13:02:06 -0700 (PDT) In-Reply-To: <20120904192320.GF88081@ithaqua.etoilebsd.net> References: <0D822C75-7240-42E3-8780-18CABC9C45A1@netasq.com> <20120903212340.GE88081@ithaqua.etoilebsd.net> <60A32BCE-756A-4A0D-A24D-7AC42F71BD1A@netasq.com> <20120904192320.GF88081@ithaqua.etoilebsd.net> Date: Tue, 4 Sep 2012 22:02:06 +0200 X-Google-Sender-Auth: BS1DuUPfjlNrcJ64gwcPn24ZA4c Message-ID: From: Davide Italiano To: Baptiste Daroussin Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-current , Fabien Thomas Subject: Re: [CFT] hwpmc support for Intel Ivy Bridge X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 20:02:07 -0000 [trimming old mails] > > Hi, > > here are the results > > # pmccontrol -L > SOFT > CLOCK.STAT > CLOCK.HARD > LOCK.FAILED > PAGE_FAULT.WRITE > PAGE_FAULT.READ > PAGE_FAULT.ALL > > # pmcstat -SOFF_CORE_RESPONSE_0,rsp=REQ_DMND_DATA_RD+RES_ANY -w1 -T > pmcstat: ERROR: Cannot allocate system-mode pmc with specification "OFF_CORE_RESPONSE_0,rsp=REQ_DMND_DATA_RD+RES_ANY": Invalid argument > > # pmcstat -n2000000 -S INSTR_RETIRED_ANY -w4 -T > pmcstat: ERROR: Cannot allocate system-mode pmc with specification "INSTR_RETIRED_ANY": Invalid argument > > # pmcstat -n2000000 -S INSTR_RETIRED_ANY -w4 -T > pmcstat: ERROR: Cannot allocate system-mode pmc with specification "INSTR_RETIRED_ANY": Invalid argument > > # pmcstat -SPAGE_FAULT.ALL -w4 -T > (This one seems to work correctly > > # pmcstat -SOFF_CORE_RESPONSE_0,rsp=REQ_DMND_DATA_RD+RES_ANY -n2000000 -SCPU_CLK_UNHALTED.THREAD_P -n2000000 -S INSTR_RETIRED_ANY -SPAGE_FAULT > pmcstat: ERROR: Cannot allocate system-mode pmc with specification "OFF_CORE_RESPONSE_0,rsp=REQ_DMND_DATA_RD+RES_ANY": Invalid argument > Are you running this test on real hardware or are you using an hypervisor? > The pmctest.py fails: > # ./test/hwpmc/pmctest.py > Traceback (most recent call last): > File "./test/hwpmc/pmctest.py", line 94, in > main() > File "./test/hwpmc/pmctest.py", line 81, in main > stdout=PIPE) > File "/usr/local/lib/python2.7/subprocess.py", line 679, in __init__ > errread, errwrite) > File "/usr/local/lib/python2.7/subprocess.py", line 1249, in _execute_child > raise child_exception > TypeError: execv() arg 2 must contain only strings > > > regards, > Bapt Yes, this is expected. You should specify a program that should be executed (to make measurements on it). I used to use 'ls' on my tests, using -p. Davide From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 20:39:43 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A92B710656B4 for ; Tue, 4 Sep 2012 20:39:43 +0000 (UTC) (envelope-from dimitry@andric.com) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id CC9C18FC1A for ; Tue, 4 Sep 2012 20:39:42 +0000 (UTC) Received: from [192.168.0.6] (spaceball.home.andric.com [192.168.0.6]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 6ACBD5C43 for ; Tue, 4 Sep 2012 22:39:41 +0200 (CEST) Message-ID: <5046670C.6050500@andric.com> Date: Tue, 04 Sep 2012 22:39:40 +0200 From: Dimitry Andric User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20120828 Thunderbird/16.0 MIME-Version: 1.0 To: freebsd-current@FreeBSD.org Content-Type: multipart/mixed; boundary="------------010504020809040102000509" Cc: Subject: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 20:39:43 -0000 This is a multi-part message in MIME format. --------------010504020809040102000509 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi all, I recently performed a series of compiler performance tests on FreeBSD 10.0-CURRENT, particularly comparing gcc 4.2.1 and gcc 4.7.1 against clang 3.1 and clang 3.2. The attached text file[1] contains more information about the tests, some semi-cooked performance data, and my conclusions. Any errors and omissions are also my fault, so if you notice them, please let me know. The executive summary: clang compiles mostly faster than gcc (sometimes much faster), and uses significantly less memory. Finally, please note these tests were purely about compilation speed, not about the performance of the resulting executables. This still needs to be tested. -Dimitry [1]: Also available at: --------------010504020809040102000509 Content-Type: text/plain; charset=windows-1252; name="perftest-2012-09-01a.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="perftest-2012-09-01a.txt" COMPILER PERFORMANCE TESTS ON FREEBSD 10.0-CURRENT, SEPTEMBER 2012 ================================================================== INTRODUCTION ------------ The compilers tested were: - gcc 4.2.1, the system compiler in FreeBSD, which is compiled by gcc 4.2.1. - gcc 4.7.1, from the official gcc.gnu.org release, compiled via a three-stage bootstrap, so the final compiler has been compiled by gcc 4.7.1. - clang 3.1 (branches/release_31 156863), which is the default version of clang in FreeBSD 10-CURRENT before r239462. The used executable was compiled by a previous copy of itself. - clang 3.2 (trunk 162107), which is the default version of clang in FreeBSD 10.0-CURRENT, after r239462. The used executable was compiled by a previous copy of itself. All tests were run on ref10-amd64.freebsd.org, which is a Dell 2950, 1.86GHz Core2 Xeon, 2x4 Core, 16G RAM. It runs FreeBSD/amd64 10.0-CURRENT #0 r231914: Sun Feb 19 17:24:37 UTC 2012. Each build was repeated 6 times, after cleaning out the object directories, and syncing. Each build was timed using the system time(1) command, using the -l argument to obtain rusage information. The programs tested by compilation were: - A large C++ program: clang 3.2, as it occurs in the FreeBSD 10.0-CURRENT source tree as of r239532. - A medium-large C program: gcc 4.2.1, as it occurs in the FreeBSD 10.0-CURRENT source tree as of r239532. - A large C++ library: boost 1.50.0, the officially released version from . Building a large C++ program (clang 3.2) single-threaded ======================================================== Using clang 3.1: ---------------- N Min Max Median Avg Stddev real 6 2283.69 2288.46 2285.74 2285.505 1.6470064 user 6 2145.2 2147.2 2146.18 2146.0567 0.68266146 sys 6 128.3 132.08 130.65 130.54833 1.256653 maxrss 6 179264 179264 179264 179264 0 ixrss 6 21407 21436 21420 21419.833 9.6211572 idrss 6 3628 3632 3630 3629.8333 1.3291601 isrss 6 252 252 252 252 0 minflt 6 12485556 12485556 12485556 12485556 0 majflt 6 0 0 0 0 0 nswap 6 0 0 0 0 0 inblock 6 0 0 0 0 0 oublock 6 2058 2106 2103 2081.3333 25.216397 msgsnd 6 18 18 18 18 0 msgrcv 6 0 0 0 0 0 nsignals 6 1878 1878 1878 1878 0 nvcsw 6 16288 16357 16333 16320.667 29.615311 nivcsw 6 2071535 3998751 3057756 2966314 635381.66 Using clang 3.2: ---------------- N Min Max Median Avg Stddev real 6 2358.61 2362.84 2362.67 2361.22 1.7831321 user 6 2215.33 2221.13 2218.72 2218.57 2.0094278 sys 6 130.78 134.63 133.41 132.99833 1.4702301 maxrss 6 177796 177796 177796 177796 0 ixrss 6 21388 21413 21408 21400.833 11.052903 idrss 6 3702 3707 3706 3704.6667 2.2509257 isrss 6 253 253 253 253 0 minflt 6 12583827 12583827 12583827 12583827 0 majflt 6 0 0 0 0 0 nswap 6 0 0 0 0 0 inblock 6 0 0 0 0 0 oublock 6 2036 2074 2071 2054.8333 19.589963 msgsnd 6 18 26 26 23.333333 4.1311822 msgrcv 6 0 0 0 0 0 nsignals 6 1878 1878 1878 1878 0 nvcsw 6 16266 16391 16354 16327.667 53.909801 nivcsw 6 2118900 3891231 3534528 3168715.7 673236.29 Using gcc 4.2.1: ---------------- N Min Max Median Avg Stddev real 6 4238.49 4241.76 4240.78 4240.1867 1.2375244 user 6 3903.48 3908.6 3907.58 3906.5583 1.8932661 sys 6 358.38 361.43 359.94 359.94667 1.1494984 maxrss 6 568592 568592 568592 568592 0 ixrss 6 6348 6353 6350 6350.3333 1.6329932 idrss 6 3495 3498 3497 3496.5 1.0488088 isrss 6 146 146 146 146 0 minflt 6 47304156 47304184 47304175 47304172 10.545141 majflt 6 0 0 0 0 0 nswap 6 0 0 0 0 0 inblock 6 0 0 0 0 0 oublock 6 2620 2754 2732 2683.6667 68.730391 msgsnd 6 0 0 0 0 0 msgrcv 6 0 0 0 0 0 nsignals 6 1878 1878 1878 1878 0 nvcsw 6 67561 67763 67674 67648 75.198404 nivcsw 6 3994087 5821442 4846679 4810028.2 597366.52 Using gcc 4.7.1: ---------------- N Min Max Median Avg Stddev real 6 3818.41 3974.54 3820.49 3846.7417 62.715466 user 6 3506.86 3591.97 3509.96 3522.8283 33.896088 sys 6 333.58 364.34 338.93 340.70833 11.839839 maxrss 6 480724 480736 480724 480727.33 5.316639 ixrss 6 12173 12198 12194 12188.333 9.4375138 idrss 6 1520 1523 1522 1521.8333 1.1690452 isrss 6 134 134 134 134 0 minflt 6 38406568 38406673 38406592 38406599 38.768544 majflt 6 0 90 0 20.333333 36.45088 nswap 6 0 0 0 0 0 inblock 6 0 4775 0 1233.3333 2028.0327 oublock 6 2266 2301 2286 2284.3333 13.662601 msgsnd 6 30 31 30 30.166667 0.40824829 msgrcv 6 0 0 0 0 0 nsignals 6 1878 1878 1878 1878 0 nvcsw 6 59792 67936 60369 61859.833 3186.3204 nivcsw 6 2867702 4546665 4361653 3753550.8 769382.51 Summary: -------- For building this specific large C++ program, gcc 4.2.1 is ~86% slower than clang 3.1 in real time, ~82% slower in user time, and ~176% slower in system time. The maximum resident set size during building is ~217% larger, and it causes ~279% more page reclaims. Though gcc 4.7.1 is faster than its older version, it is still ~68% slower than clang 3.1 in real time, ~64% slower in user time, and ~161% slower in system time. The maximum resident set size during building is ~220% larger, and it causes ~208% more page reclaims. Finally, clang 3.2 is ~3% slower than clang 3.1 in both real time and user time, and ~2% slower in system time. The maximum resident set size and the number of page reclaims during building are approximately equal. Conclusion: ----------- Clang 3.1 is clearly the fastest compiler for building this specific large C++ program, with clang 3.2 trailing closely behind. Both are significantly faster, and use much less memory than either version of gcc. Building a medium-large C program (gcc 4.2.1) single-threaded ============================================================= Using clang 3.1: ---------------- N Min Max Median Avg Stddev real 6 303.31 304.06 303.65 303.67167 0.24991332 user 6 275.42 277.09 275.99 276.11167 0.57766484 sys 6 24.92 26.15 25.6 25.656667 0.44643775 maxrss 6 177876 177876 177876 177876 0 ixrss 6 20529 20559 20544 20542.833 12.38413 idrss 6 3618 3623 3621 3620.3333 1.9663842 isrss 6 247 247 247 247 0 minflt 6 2214250 2214250 2214250 2214250 0 majflt 6 0 0 0 0 0 nswap 6 0 0 0 0 0 inblock 6 0 0 0 0 0 oublock 6 677 677 677 677 0 msgsnd 6 18 18 18 18 0 msgrcv 6 0 0 0 0 0 nsignals 6 883 883 883 883 0 nvcsw 6 5705 5837 5819 5793.6667 49.33018 nivcsw 6 205418 467152 449398 371699.67 114414.58 Using clang 3.2: ---------------- N Min Max Median Avg Stddev real 6 330.22 331.23 330.95 330.69833 0.43687145 user 6 301.29 302.59 302.3 302.05667 0.49649438 sys 6 26.12 27.19 27.06 26.875 0.39747956 maxrss 6 186260 186260 186260 186260 0 ixrss 6 20639 20674 20660 20656.833 14.469508 idrss 6 3699 3705 3703 3702.3333 2.1602469 isrss 6 316 319 318 317.33333 1.2110601 minflt 6 2290933 2290934 2290934 2290933.7 0.51614557 majflt 6 0 0 0 0 0 nswap 6 0 0 0 0 0 inblock 6 0 0 0 0 0 oublock 6 668 669 668 668.16667 0.40824829 msgsnd 6 18 18 18 18 0 msgrcv 6 0 0 0 0 0 nsignals 6 883 883 883 883 0 nvcsw 6 5783 5822 5801 5799 17.944358 nivcsw 6 111115 520961 396082 316725.33 164041.32 Using gcc 4.2.1: ---------------- N Min Max Median Avg Stddev real 6 422.68 425.44 423.23 423.47333 1.0273396 user 6 389.1 391.67 390.58 390.41333 0.82734918 sys 6 36.85 39.2 38.65 38.23 0.83840324 maxrss 6 392560 392560 392560 392560 0 ixrss 6 5529 5542 5534 5534.5 6.0580525 idrss 6 3915 3924 3919 3919 4.1472883 isrss 6 142 142 142 142 0 minflt 6 4055461 4055464 4055463 4055462.7 1.21063 majflt 6 0 4 0 0.66666667 1.6329932 nswap 6 0 0 0 0 0 inblock 6 0 730 0 121.66667 298.02125 oublock 6 659 693 662 667 12.884099 msgsnd 6 0 0 0 0 0 msgrcv 6 0 0 0 0 0 nsignals 6 883 883 883 883 0 nvcsw 6 15645 16454 15874 15888 298.36019 nivcsw 6 121293 661776 414611 363556.83 207101.28 Using gcc 4.7.1: ---------------- N Min Max Median Avg Stddev real 6 461.58 462.55 462.01 461.98333 0.40287302 user 6 425.22 426.36 425.92 425.835 0.43825791 sys 6 40.83 42.94 41.99 41.925 0.71034499 maxrss 6 445624 445624 445624 445624 0 ixrss 6 10781 10816 10801 10797.5 12.405644 idrss 6 2427 2433 2430 2430 2.1908902 isrss 6 178 178 178 178 0 minflt 6 3883735 3883740 3883739 3883738 2.3664319 majflt 6 0 0 0 0 0 nswap 6 0 0 0 0 0 inblock 6 0 14 0 2.3333333 5.7154761 oublock 6 677 681 679 679 1.4142136 msgsnd 6 20 20 20 20 0 msgrcv 6 0 0 0 0 0 nsignals 6 883 883 883 883 0 nvcsw 6 16411 16660 16532 16542.333 98.544744 nivcsw 6 284414 901533 384379 449845.33 241447.41 Summary: -------- For building this specific medium C program, gcc 4.2.1 is ~40% slower than clang 3.1 in real time, ~41% slower in user time, and ~49% slower in system time. The maximum resident set size during building is ~121% larger, and it causes ~83% more page reclaims. For C, gcc 4.7.1 is even slower than its older version; it is ~52% slower than clang 3.1 in real time, ~54% slower in user time, and ~63% slower in system time. The maximum resident set size during building is ~151% larger, and it causes ~75% more page reclaims. Finally, clang 3.2 is ~9% slower than clang 3.1 in both real time and user time, and ~5% slower in system time. The maximum resident set size during building is ~5% larger, and it causes ~4% more page reclaims. Conclusion: ----------- Clang 3.1 is clearly the fastest compiler for building this specific medium- large C program, with clang 3.2 somewhat behind. Both are significantly faster, and use much less memory than either version of gcc. Building a large C++ library (boost 1.50.0) single-threaded =========================================================== Using clang 3.1: ---------------- N Min Max Median Avg Stddev real 6 1056.69 1060.49 1059.09 1058.6783 1.5028695 user 6 975.49 978.88 978.53 977.55 1.4653464 sys 6 73.75 76.42 74.87 74.95 1.0609618 maxrss 6 212324 216712 213668 214260.67 1774.6309 ixrss 6 22472 22549 22525 22514.5 31.232995 idrss 6 3793 3806 3802 3800.1667 5.492419 isrss 6 276 277 277 276.5 0.54772256 minflt 6 9543701 9543702 9543701 9543701.3 0.51234754 majflt 6 0 0 0 0 0 nswap 6 0 0 0 0 0 inblock 6 0 0 0 0 0 oublock 6 1453 1461 1457 1455.8333 3.3714487 msgsnd 6 115 115 115 115 0 msgrcv 6 0 0 0 0 0 nsignals 6 0 0 0 0 0 nvcsw 6 7352 7834 7576 7567.1667 167.70023 nivcsw 6 27478 2350999 1699745 1337037.8 1040439 Using clang 3.2: ---------------- N Min Max Median Avg Stddev real 6 1075.33 1077.94 1076.39 1076.4267 0.93958856 user 6 995.14 997.61 995.43 995.88833 0.9489661 sys 6 72.34 74.67 74.23 73.843333 0.81563881 maxrss 6 208552 211148 210436 209936 921.08458 ixrss 6 22437 22484 22458 22459 19.768662 idrss 6 3869 3878 3873 3873.3333 3.8815804 isrss 6 275 275 275 275 0 minflt 6 9351477 9351478 9351478 9351477.5 0.54772256 majflt 6 0 0 0 0 0 nswap 6 0 0 0 0 0 inblock 6 0 0 0 0 0 oublock 6 1448 1454 1449 1449.6667 2.4221203 msgsnd 6 115 115 115 115 0 msgrcv 6 0 0 0 0 0 nsignals 6 0 0 0 0 0 nvcsw 6 10481 12934 11049 11105.333 936.9249 nivcsw 6 975292 2383586 1633650 1615797.3 605542.76 Using gcc 4.2.1: ---------------- N Min Max Median Avg Stddev real 6 1037.86 1047.78 1039.71 1040.21 3.8054592 user 6 938.74 944.49 941.52 941.55667 1.8382999 sys 6 86.37 92.84 89.89 89.57 2.1105639 maxrss 6 560256 560316 560272 560274 21.428952 ixrss 6 6435 6453 6441 6443 7.2663608 idrss 6 3563 3573 3566 3567.5 4.0373258 isrss 6 136 136 136 136 0 minflt 6 12360490 12360492 12360491 12360491 0.63245553 majflt 6 0 0 0 0 0 nswap 6 0 0 0 0 0 inblock 6 0 4283 0 713.83333 1748.5274 oublock 6 2648 2656 2655 2653.3333 2.9439203 msgsnd 6 44 51 51 49.833333 2.857738 msgrcv 6 0 0 0 0 0 nsignals 6 0 0 0 0 0 nvcsw 6 7897 12281 8004 8696 1757.1741 nivcsw 6 19915 1989580 897003 957452.5 764999.83 Using gcc 4.7.1: ---------------- N Min Max Median Avg Stddev real 6 1038.13 1041.29 1040.98 1039.92 1.3837774 user 6 937.73 943.59 941.35 941.14167 2.0323919 sys 6 89.19 95.1 91.61 91.588333 2.0745739 maxrss 6 361268 361268 361268 361268 0 ixrss 6 12431 12474 12469 12457.333 17.385818 idrss 6 1547 1552 1551 1549.8333 1.9407902 isrss 6 129 129 129 129 0 minflt 6 10455489 10455489 10455489 10455489 0 majflt 6 0 0 0 0 0 nswap 6 0 0 0 0 0 inblock 6 0 162 0 27 66.136223 oublock 6 2537 2540 2539 2538.5 1.0488088 msgsnd 6 113 113 113 113 0 msgrcv 6 0 0 0 0 0 nsignals 6 0 0 0 0 0 nvcsw 6 7778 7975 7880 7874.1667 78.036957 nivcsw 6 27055 2302383 2187401 1478808.3 946760.86 Summary: -------- For building this specific large C++ library, clang 3.1 is ~2% slower than gcc 4.2.1 in real time, ~4% slower in user time, but ~20% faster in system time. The maximum resident set size during building is ~162% smaller, and it causes ~30% less page reclaims. As before, clang 3.2 is slower than its older version; it is ~3% slower than gcc 4.2.1 in real time, ~6% slower in user time, but ~21% faster in system time. The maximum resident set size is ~167% smaller, and it causes ~32% less page reclaims. Finally, gcc 4.7.1 is equally fast as gcc 4.2.1 in real time and user time, and ~2% slower in system time. The maximum resident set size is ~36% smaller, and it causes ~15% less page reclaims. Conclusion: ----------- Both gcc 4.2.1 and 4.7.1 are the fastest compilers for building this specific large C++ library, but both versions of clang are not far behind. Both versions of gcc use quite a bit more memory than either version of clang. ================================================================================ Copyright (c) 2012 Dimitry Andric Verbatim copying and redistribution of this entire text are permitted, provided this notice is preserved. ================================================================================ --------------010504020809040102000509-- From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 20:56:35 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BDD9D10656EB for ; Tue, 4 Sep 2012 20:56:34 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id 57B9A8FC21 for ; Tue, 4 Sep 2012 20:56:34 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.69) with esmtp (envelope-from ) id <1T90Ac-00026w-MZ>; Tue, 04 Sep 2012 22:56:26 +0200 Received: from e178032096.adsl.alicedsl.de ([85.178.32.96] helo=thor.walstatt.dyndns.org) by inpost2.zedat.fu-berlin.de (Exim 4.69) with esmtpsa (envelope-from ) id <1T90Ac-0002tF-Hn>; Tue, 04 Sep 2012 22:56:26 +0200 Message-ID: <50466AF9.5010509@zedat.fu-berlin.de> Date: Tue, 04 Sep 2012 22:56:25 +0200 From: "O. Hartmann" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120810 Thunderbird/14.0 MIME-Version: 1.0 To: Dimitry Andric References: <5046670C.6050500@andric.com> In-Reply-To: <5046670C.6050500@andric.com> X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig2CF9B7BF134CCED0B87A830B" X-Originating-IP: 85.178.32.96 Cc: freebsd-current@FreeBSD.org Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 20:56:35 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig2CF9B7BF134CCED0B87A830B Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 09/04/12 22:39, Dimitry Andric wrote: > Hi all, >=20 > I recently performed a series of compiler performance tests on FreeBSD > 10.0-CURRENT, particularly comparing gcc 4.2.1 and gcc 4.7.1 against > clang 3.1 and clang 3.2. >=20 > The attached text file[1] contains more information about the tests, > some semi-cooked performance data, and my conclusions. Any errors and > omissions are also my fault, so if you notice them, please let me know.= >=20 > The executive summary: clang compiles mostly faster than gcc (sometimes= > much faster), and uses significantly less memory. >=20 > Finally, please note these tests were purely about compilation speed, > not about the performance of the resulting executables. This still > needs to be tested. >=20 > -Dimitry >=20 > [1]: Also available at: > Very intersting. It would also be of great interest to have some benchmarks on FBSD 10 at hand which compare the performance of the resulting binary of those compilers. Regards, Oliver --------------enig2CF9B7BF134CCED0B87A830B Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBAgAGBQJQRmr5AAoJEOgBcD7A/5N8BHMH/jynI19T0SZvbEjNY5LHBS16 xvcICPrHbVvx+tx9CjY0zSK5pWC1ouEDjfdnfLI+F+gIH4m44e6J6WCjnFDVfFzA csixg30mpHTuTJ7VvwA94guCDqDNEjrb2vFjn73TYH5X5W45UsJ9EBDEUubdofk1 CkXnTx4A/lJ6ApxmN6hq0AwAjgTv9py+cTDIF8Z5X3r5/+VwQa6PSwKS7ZOocwIT LmnA+9gLGI2vcZ3ZXZRwpQuH9AAMTJjvW1Ph5/i3CICZZcWjm3kRIqrSLaTBSAoO AAyvIHcdwn0lX1gaamxnCumeDEyeHWaIn/DZ399MT7xuNPTXa09elB1AQNlf5SM= =e5iu -----END PGP SIGNATURE----- --------------enig2CF9B7BF134CCED0B87A830B-- From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 21:43:55 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 569811065670 for ; Tue, 4 Sep 2012 21:43:55 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) by mx1.freebsd.org (Postfix) with ESMTP id 340208FC14 for ; Tue, 4 Sep 2012 21:43:55 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.5/8.14.5) with ESMTP id q84LhiZs018450; Tue, 4 Sep 2012 14:43:44 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.5/8.14.5/Submit) id q84LhiVJ018449; Tue, 4 Sep 2012 14:43:44 -0700 (PDT) (envelope-from sgk) Date: Tue, 4 Sep 2012 14:43:44 -0700 From: Steve Kargl To: Dimitry Andric Message-ID: <20120904214344.GA17723@troutmask.apl.washington.edu> References: <5046670C.6050500@andric.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5046670C.6050500@andric.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-current@freebsd.org Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 21:43:55 -0000 On Tue, Sep 04, 2012 at 10:39:40PM +0200, Dimitry Andric wrote: > > I recently performed a series of compiler performance tests on FreeBSD > 10.0-CURRENT, particularly comparing gcc 4.2.1 and gcc 4.7.1 against > clang 3.1 and clang 3.2. > > The attached text file[1] contains more information about the tests, > some semi-cooked performance data, and my conclusions. Any errors and > omissions are also my fault, so if you notice them, please let me know. > > The executive summary: clang compiles mostly faster than gcc (sometimes > much faster), and uses significantly less memory. The benchmark is somewhat meaningless if one does not know the options that were used during the testing. -- Steve From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 21:50:57 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 840A7106583F for ; Tue, 4 Sep 2012 21:50:57 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 342B18FC08 for ; Tue, 4 Sep 2012 21:50:57 +0000 (UTC) Received: by iayy25 with SMTP id y25so67444iay.13 for ; Tue, 04 Sep 2012 14:50:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Rwv0uF+g1ipa5D/ajoZ6LEo6ehabx2kVpNIrt2DPHCs=; b=G51Fd9/4cxMWnPn8JfY9tBYWTu3E4JzyByarY63I4PIW1nkQN17NH8HWSa6/GlD5f9 iz2ZUqJ0zXrQZ0+4VsuPMmcXhBmkbTsexIytLAqTp6pBNq6YZYEuDEsLom4uXSysoGyo P7QFZe9xeXrEfb9+uETe7Ut8gAbhRnOUh2cwLhXbBd0AiMTL18fMp8ZuqtAnmYjnZ2V1 +MvsZR6qeIZ2ErJ2KP7qfgqdIet68nryQQKyVn6ezmJX42CPBx+Z8J8bgk/KLHYAAox5 XfCzNNG9mumF9t/FhNdKF2L9Z0YwU2zqLSnLtRRSZGYdVQqVUT7BjQrWX5GMYpFPddV7 zv3A== MIME-Version: 1.0 Received: by 10.60.2.134 with SMTP id 6mr16359633oeu.62.1346795456637; Tue, 04 Sep 2012 14:50:56 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Tue, 4 Sep 2012 14:50:56 -0700 (PDT) In-Reply-To: <5046670C.6050500@andric.com> References: <5046670C.6050500@andric.com> Date: Tue, 4 Sep 2012 14:50:56 -0700 Message-ID: From: Garrett Cooper To: Dimitry Andric Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-current@freebsd.org Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 21:50:57 -0000 On Tue, Sep 4, 2012 at 1:39 PM, Dimitry Andric wrote: > Hi all, > > I recently performed a series of compiler performance tests on FreeBSD > 10.0-CURRENT, particularly comparing gcc 4.2.1 and gcc 4.7.1 against > clang 3.1 and clang 3.2. > > The attached text file[1] contains more information about the tests, > some semi-cooked performance data, and my conclusions. Any errors and > omissions are also my fault, so if you notice them, please let me know. > > The executive summary: clang compiles mostly faster than gcc (sometimes > much faster), and uses significantly less memory. > > Finally, please note these tests were purely about compilation speed, > not about the performance of the resulting executables. This still > needs to be tested. It would be interesting to see how clang++ performs vs g++ when dealing with nested classes and with complicated code when trying to optimize things because the optimizer in g++ apparently has some scaling issues. Thanks! -Garrett From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 21:55:47 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 98EBC1065673; Tue, 4 Sep 2012 21:55:47 +0000 (UTC) (envelope-from bapt@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 780FF8FC19; Tue, 4 Sep 2012 21:55:47 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q84LtlI4051661; Tue, 4 Sep 2012 21:55:47 GMT (envelope-from bapt@freebsd.org) Received: (from bapt@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q84Ltlct051632; Tue, 4 Sep 2012 21:55:47 GMT (envelope-from bapt@freebsd.org) X-Authentication-Warning: freefall.freebsd.org: bapt set sender to bapt@freebsd.org using -f Date: Tue, 4 Sep 2012 23:55:44 +0200 From: Baptiste Daroussin To: Davide Italiano Message-ID: <20120904215544.GG88081@ithaqua.etoilebsd.net> References: <0D822C75-7240-42E3-8780-18CABC9C45A1@netasq.com> <20120903212340.GE88081@ithaqua.etoilebsd.net> <60A32BCE-756A-4A0D-A24D-7AC42F71BD1A@netasq.com> <20120904192320.GF88081@ithaqua.etoilebsd.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Cp3Cp8fzgozWLBWL" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-current , Fabien Thomas Subject: Re: [CFT] hwpmc support for Intel Ivy Bridge X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 21:55:47 -0000 --Cp3Cp8fzgozWLBWL Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Sep 04, 2012 at 10:02:06PM +0200, Davide Italiano wrote: > [trimming old mails] >=20 > > > > Hi, > > > > here are the results > > > > # pmccontrol -L > > SOFT > > CLOCK.STAT > > CLOCK.HARD > > LOCK.FAILED > > PAGE_FAULT.WRITE > > PAGE_FAULT.READ > > PAGE_FAULT.ALL > > > > # pmcstat -SOFF_CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY -w1 -T > > pmcstat: ERROR: Cannot allocate system-mode pmc with specification "OFF= _CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY": Invalid argument > > > > # pmcstat -n2000000 -S INSTR_RETIRED_ANY -w4 -T > > pmcstat: ERROR: Cannot allocate system-mode pmc with specification "INS= TR_RETIRED_ANY": Invalid argument > > > > # pmcstat -n2000000 -S INSTR_RETIRED_ANY -w4 -T > > pmcstat: ERROR: Cannot allocate system-mode pmc with specification "INS= TR_RETIRED_ANY": Invalid argument > > > > # pmcstat -SPAGE_FAULT.ALL -w4 -T > > (This one seems to work correctly > > > > # pmcstat -SOFF_CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY -n200000= 0 -SCPU_CLK_UNHALTED.THREAD_P -n2000000 -S INSTR_RETIRED_ANY -SPAGE_FAULT > > pmcstat: ERROR: Cannot allocate system-mode pmc with specification "OFF= _CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY": Invalid argument > > >=20 > Are you running this test on real hardware or are you using an hypervisor? >=20 > > The pmctest.py fails: > > # ./test/hwpmc/pmctest.py > > Traceback (most recent call last): > > File "./test/hwpmc/pmctest.py", line 94, in > > main() > > File "./test/hwpmc/pmctest.py", line 81, in main > > stdout=3DPIPE) > > File "/usr/local/lib/python2.7/subprocess.py", line 679, in __init__ > > errread, errwrite) > > File "/usr/local/lib/python2.7/subprocess.py", line 1249, in _execute= _child > > raise child_exception > > TypeError: execv() arg 2 must contain only strings > > > > > > regards, > > Bapt >=20 > Yes, this is expected. You should specify a program that should be > executed (to make measurements on it). I used to use 'ls' on my tests, > using -p. >=20 > Davide ./test/hwpmc/pmctest.py -p ls pmcstat: ERROR: Cannot allocate process-mode pmc with specification "SOFT": Invalid argument In that case. Sorry I don't know anything about pmc, just willing to help testing :) regards, Bapt --Cp3Cp8fzgozWLBWL Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlBGeOAACgkQ8kTtMUmk6ExPQQCgpmRPvKU9eSMCfpWhf1q4regm j1sAn3h2Awl1+I4Hp4bmKOIK4dyeFFYl =v4sk -----END PGP SIGNATURE----- --Cp3Cp8fzgozWLBWL-- From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 21:58:33 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CEDCF1065793; Tue, 4 Sep 2012 21:58:33 +0000 (UTC) (envelope-from tinderbox@freebsd.org) Received: from freebsd-current.sentex.ca (freebsd-current.sentex.ca [64.7.128.98]) by mx1.freebsd.org (Postfix) with ESMTP id 96C3D8FC17; Tue, 4 Sep 2012 21:58:33 +0000 (UTC) Received: from freebsd-current.sentex.ca (localhost [127.0.0.1]) by freebsd-current.sentex.ca (8.14.5/8.14.5) with ESMTP id q84LwRMS031644; Tue, 4 Sep 2012 17:58:27 -0400 (EDT) (envelope-from tinderbox@freebsd.org) Received: (from tinderbox@localhost) by freebsd-current.sentex.ca (8.14.5/8.14.5/Submit) id q84LwQTQ031640; Tue, 4 Sep 2012 21:58:26 GMT (envelope-from tinderbox@freebsd.org) Date: Tue, 4 Sep 2012 21:58:26 GMT Message-Id: <201209042158.q84LwQTQ031640@freebsd-current.sentex.ca> X-Authentication-Warning: freebsd-current.sentex.ca: tinderbox set sender to FreeBSD Tinderbox using -f Sender: FreeBSD Tinderbox From: FreeBSD Tinderbox To: FreeBSD Tinderbox , , Precedence: bulk Cc: Subject: [head tinderbox] failure on sparc64/sparc64 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 21:58:34 -0000 TB --- 2012-09-04 21:47:04 - tinderbox 2.9 running on freebsd-current.sentex.ca TB --- 2012-09-04 21:47:04 - FreeBSD freebsd-current.sentex.ca 8.3-PRERELEASE FreeBSD 8.3-PRERELEASE #0: Mon Mar 26 13:54:12 EDT 2012 des@freebsd-current.sentex.ca:/usr/obj/usr/src/sys/GENERIC amd64 TB --- 2012-09-04 21:47:04 - starting HEAD tinderbox run for sparc64/sparc64 TB --- 2012-09-04 21:47:04 - cleaning the object tree TB --- 2012-09-04 21:47:04 - cvsupping the source tree TB --- 2012-09-04 21:47:04 - /usr/bin/csup -z -r 3 -g -L 1 -h cvsup.sentex.ca /tinderbox/HEAD/sparc64/sparc64/supfile TB --- 2012-09-04 21:48:10 - building world TB --- 2012-09-04 21:48:10 - CROSS_BUILD_TESTING=YES TB --- 2012-09-04 21:48:10 - MAKEOBJDIRPREFIX=/obj TB --- 2012-09-04 21:48:10 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2012-09-04 21:48:10 - SRCCONF=/dev/null TB --- 2012-09-04 21:48:10 - TARGET=sparc64 TB --- 2012-09-04 21:48:10 - TARGET_ARCH=sparc64 TB --- 2012-09-04 21:48:10 - TZ=UTC TB --- 2012-09-04 21:48:10 - __MAKE_CONF=/dev/null TB --- 2012-09-04 21:48:10 - cd /src TB --- 2012-09-04 21:48:10 - /usr/bin/make -B buildworld >>> World build started on Tue Sep 4 21:48:11 UTC 2012 >>> Rebuilding the temporary build tree >>> stage 1.1: legacy release compatibility shims >>> stage 1.2: bootstrap tools >>> stage 2.1: cleaning up the object tree >>> stage 2.2: rebuilding the object tree >>> stage 2.3: build tools >>> stage 3: cross tools >>> stage 4.1: building includes >>> stage 4.2: building libraries [...] cc -O2 -pipe -I/src/lib/libc/include -I/src/lib/libc/../../include -I/src/lib/libc/sparc64 -DNLS -I/src/lib/libc/sparc64/sys -D__DBINTERFACE_PRIVATE -I/src/lib/libc/../../contrib/gdtoa -DINET6 -I/obj/sparc64.sparc64/src/lib/libc -I/src/lib/libc/resolv -D_ACL_PRIVATE -DPOSIX_MISTAKE -I/src/lib/libc/../../contrib/jemalloc/include -I/src/lib/libc/../../contrib/tzcode/stdtime -I/src/lib/libc/stdtime -I/src/lib/libc/locale -I/src/lib/libc/sparc64/fpu -DBROKEN_DES -DPORTMAP -DDES_BUILTIN -I/src/lib/libc/rpc -DYP -DNS_CACHING -DSYMBOL_VERSIONING -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -Wno-uninitialized -Wno-pointer-sign -c /src/lib/libc/net/gai_strerror.c -o gai_strerror.o cc -O2 -pipe -I/src/lib/libc/include -I/src/lib/libc/../../include -I/src/lib/libc/sparc64 -DNLS -I/src/lib/libc/sparc64/sys -D__DBINTERFACE_PRIVATE -I/src/lib/libc/../../contrib/gdtoa -DINET6 -I/obj/sparc64.sparc64/src/lib/libc -I/src/lib/libc/resolv -D_ACL_PRIVATE -DPOSIX_MISTAKE -I/src/lib/libc/../../contrib/jemalloc/include -I/src/lib/libc/../../contrib/tzcode/stdtime -I/src/lib/libc/stdtime -I/src/lib/libc/locale -I/src/lib/libc/sparc64/fpu -DBROKEN_DES -DPORTMAP -DDES_BUILTIN -I/src/lib/libc/rpc -DYP -DNS_CACHING -DSYMBOL_VERSIONING -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -Wno-uninitialized -Wno-pointer-sign -c /src/lib/libc/net/getaddrinfo.c -o getaddrinfo.o In file included from /src/lib/libc/net/getaddrinfo.c:67: /obj/sparc64.sparc64/src/tmp/usr/include/net/if_var.h:102: error: expected declaration specifiers or '...' before 'link_pfil_hook' cc1: warnings being treated as errors /obj/sparc64.sparc64/src/tmp/usr/include/net/if_var.h:102: warning: 'struct pfil_head' declared inside parameter list /obj/sparc64.sparc64/src/tmp/usr/include/net/if_var.h:102: warning: its scope is only this definition or declaration, which is probably not what you want /obj/sparc64.sparc64/src/tmp/usr/include/net/if_var.h:102: warning: data definition has no type or storage class *** Error code 1 Stop in /src/lib/libc. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. TB --- 2012-09-04 21:58:26 - WARNING: /usr/bin/make returned exit code 1 TB --- 2012-09-04 21:58:26 - ERROR: failed to build world TB --- 2012-09-04 21:58:26 - 463.31 user 74.46 system 682.41 real http://tinderbox.freebsd.org/tinderbox-head-HEAD-sparc64-sparc64.full From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 21:59:48 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07D8910656D8 for ; Tue, 4 Sep 2012 21:59:48 +0000 (UTC) (envelope-from dimitry@andric.com) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) by mx1.freebsd.org (Postfix) with ESMTP id B6DE58FC16 for ; Tue, 4 Sep 2012 21:59:47 +0000 (UTC) Received: from [192.168.0.6] (spaceball.home.andric.com [192.168.0.6]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 255F75C44; Tue, 4 Sep 2012 23:59:41 +0200 (CEST) Message-ID: <504679CB.90204@andric.com> Date: Tue, 04 Sep 2012 23:59:39 +0200 From: Dimitry Andric User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20120828 Thunderbird/16.0 MIME-Version: 1.0 To: Steve Kargl References: <5046670C.6050500@andric.com> <20120904214344.GA17723@troutmask.apl.washington.edu> In-Reply-To: <20120904214344.GA17723@troutmask.apl.washington.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 21:59:48 -0000 On 2012-09-04 23:43, Steve Kargl wrote: > On Tue, Sep 04, 2012 at 10:39:40PM +0200, Dimitry Andric wrote: >> I recently performed a series of compiler performance tests on FreeBSD >> 10.0-CURRENT, particularly comparing gcc 4.2.1 and gcc 4.7.1 against >> clang 3.1 and clang 3.2. ... > The benchmark is somewhat meaningless if one does not > know the options that were used during the testing. If you meant the compilation options, those were simply the FreeBSD defaults for all tested programs, e.g. "-O2 -pipe", except for boost, which uses "-ftemplate-depth-128 -O3 -finline-functions". I will add some explicit notes about them. From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 22:14:18 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EADE5106566B for ; Tue, 4 Sep 2012 22:14:18 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) by mx1.freebsd.org (Postfix) with ESMTP id C72718FC16 for ; Tue, 4 Sep 2012 22:14:18 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.5/8.14.5) with ESMTP id q84MEDo9019513; Tue, 4 Sep 2012 15:14:13 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.5/8.14.5/Submit) id q84MEDfs019512; Tue, 4 Sep 2012 15:14:13 -0700 (PDT) (envelope-from sgk) Date: Tue, 4 Sep 2012 15:14:13 -0700 From: Steve Kargl To: Dimitry Andric Message-ID: <20120904221413.GA19395@troutmask.apl.washington.edu> References: <5046670C.6050500@andric.com> <20120904214344.GA17723@troutmask.apl.washington.edu> <504679CB.90204@andric.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <504679CB.90204@andric.com> User-Agent: Mutt/1.4.2.3i Cc: freebsd-current@freebsd.org Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 22:14:19 -0000 On Tue, Sep 04, 2012 at 11:59:39PM +0200, Dimitry Andric wrote: > On 2012-09-04 23:43, Steve Kargl wrote: > >On Tue, Sep 04, 2012 at 10:39:40PM +0200, Dimitry Andric wrote: > >>I recently performed a series of compiler performance tests on FreeBSD > >>10.0-CURRENT, particularly comparing gcc 4.2.1 and gcc 4.7.1 against > >>clang 3.1 and clang 3.2. > ... > >The benchmark is somewhat meaningless if one does not > >know the options that were used during the testing. > > If you meant the compilation options, those were simply the FreeBSD > defaults for all tested programs, e.g. "-O2 -pipe", except for boost, > which uses "-ftemplate-depth-128 -O3 -finline-functions". I will add > some explicit notes about them. Yes, I meant the options specified on the compiler command line. 'gcc -O0 -pipe' compiles code faster than 'gcc -O3 -save-temps', and the former uses much less memory. -- Steve From owner-freebsd-current@FreeBSD.ORG Tue Sep 4 23:40:56 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0920F106566B for ; Tue, 4 Sep 2012 23:40:56 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id C44028FC12 for ; Tue, 4 Sep 2012 23:40:55 +0000 (UTC) Received: by iayy25 with SMTP id y25so245218iay.13 for ; Tue, 04 Sep 2012 16:40:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=75yr5/MhiRfkRX3fAR6S5/cnSO0xti0JtzKxgyGyH5c=; b=GEWPD5EOr4U6jm2ITaUeIZvkTiX4Y/aeJnMPMLAd03HbUw/K0UK490Zyqo49JyL3yT /EYqJc1z6euFslEEMKPZ2liXe6TZ0U6iM3Aic/sPReCgP/oQShXsYtQFQoeJJlIZRWPE Y6gvPM1Lg66gMjA5Xbck+h8IHtdGwVn1cShYqSZG/lzGiwwZVdcegOhY8hKQuk7dhjIn MAg1pIcgps36qytsQ5f2Sjkk7KuhkVpV0HuNGvkaJ8YaIj5pfHYMLpFa1Gpq61mEiuL1 vLesQS0uznmGfk9kCSdKe0XA2kUBveI3wZ6d18dSU6oMzPvlfQK+pPMrpgkOHVZYISI2 WIcQ== MIME-Version: 1.0 Received: by 10.60.2.134 with SMTP id 6mr16411317oeu.62.1346802055134; Tue, 04 Sep 2012 16:40:55 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Tue, 4 Sep 2012 16:40:55 -0700 (PDT) In-Reply-To: <20120904221413.GA19395@troutmask.apl.washington.edu> References: <5046670C.6050500@andric.com> <20120904214344.GA17723@troutmask.apl.washington.edu> <504679CB.90204@andric.com> <20120904221413.GA19395@troutmask.apl.washington.edu> Date: Tue, 4 Sep 2012 16:40:55 -0700 Message-ID: From: Garrett Cooper To: Steve Kargl Content-Type: text/plain; charset=ISO-8859-1 Cc: Dimitry Andric , freebsd-current@freebsd.org Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Sep 2012 23:40:56 -0000 On Tue, Sep 4, 2012 at 3:14 PM, Steve Kargl wrote: > On Tue, Sep 04, 2012 at 11:59:39PM +0200, Dimitry Andric wrote: >> On 2012-09-04 23:43, Steve Kargl wrote: >> >On Tue, Sep 04, 2012 at 10:39:40PM +0200, Dimitry Andric wrote: >> >>I recently performed a series of compiler performance tests on FreeBSD >> >>10.0-CURRENT, particularly comparing gcc 4.2.1 and gcc 4.7.1 against >> >>clang 3.1 and clang 3.2. >> ... >> >The benchmark is somewhat meaningless if one does not >> >know the options that were used during the testing. >> >> If you meant the compilation options, those were simply the FreeBSD >> defaults for all tested programs, e.g. "-O2 -pipe", except for boost, >> which uses "-ftemplate-depth-128 -O3 -finline-functions". I will add >> some explicit notes about them. > > Yes, I meant the options specified on the compiler command line. > 'gcc -O0 -pipe' compiles code faster than 'gcc -O3 -save-temps', > and the former uses much less memory. Steve does have a point. Posting the results of CFLAGS/CPPFLAGS/LDFLAGS/etc for config.log (and maybe poking through the code to figure out what *FLAGS were used elsewhere) is more valuable than the data is in its current state (unfortunately.. autoconf makes things more complicated). Maybe we need some micro benchmarks for this (no, I'm not volunteering :P). Thanks! -Garrett From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 07:02:12 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 58395106564A for ; Wed, 5 Sep 2012 07:02:12 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 8FE648FC0C for ; Wed, 5 Sep 2012 07:02:11 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA13141; Wed, 05 Sep 2012 10:02:09 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1T99cm-0002OM-J4; Wed, 05 Sep 2012 10:02:08 +0300 Message-ID: <5046F8EF.7070403@FreeBSD.org> Date: Wed, 05 Sep 2012 10:02:07 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120901 Thunderbird/15.0 MIME-Version: 1.0 To: Ryan Stone References: In-Reply-To: X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: FreeBSD Current Subject: Re: machdep.hyperthreading_allowed does not affect SMT cores X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 07:02:12 -0000 on 04/09/2012 19:01 Ryan Stone said the following: > I have a Intel Sandy Bridge system that reports that it has SMT cores > instead of HTT(under a derivative of FreeBSD 8.2). I'll admit that I > don't at all understand the distinction between the two -- I thought > that HTT was just Intel's name for SMT. In any case, is there any > reason that machdep.hyperthreading_allowed should not apply to SMT > cores, too? I tend to agree with you. What we have now is a historic artifact. For reasons, that are irrelevant now, "HTT" is used as a code name for "old" Intel's SMT aka HTT and "SMT" is used for "newer" Intel's SMT aka HTT. It was considered that old HTT was "bad and ugly" and new HTT was good. So all kinds of limitations were applied to the old one and not to the new one. While there is certain truth to that distinction, what FreeBSD currently does is still illogical. HTT is still HTT whether it's new or old. E.g. I think that it doesn't make sense to assign interrupts to the second thread with either version of HTT. Besides, I think that the current CPU detection code sucks at deciding which HTT is which. I have some new SMP topology detection code in the works. It eliminates the artificial difference and retires "HTT" name in favor of the vendor-agnostic "SMT". I just a final push to tidy up that code before suggesting it for the tree. But I wasn't able to make that push in over a year, sigh... -- Andriy Gapon From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 07:33:56 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0C7D91065676; Wed, 5 Sep 2012 07:33:56 +0000 (UTC) (envelope-from fabien.thomas@netasq.com) Received: from work.netasq.com (gwlille.netasq.com [91.212.116.1]) by mx1.freebsd.org (Postfix) with ESMTP id 501578FC08; Wed, 5 Sep 2012 07:33:55 +0000 (UTC) Received: from [10.2.1.1] (unknown [10.2.1.1]) by work.netasq.com (Postfix) with ESMTPSA id 8A4132704475; Wed, 5 Sep 2012 09:33:54 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=iso-8859-1 From: Fabien Thomas In-Reply-To: <20120904192320.GF88081@ithaqua.etoilebsd.net> Date: Wed, 5 Sep 2012 09:33:55 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <441C9DEC-517B-4991-A2F9-33E8080C0B19@netasq.com> References: <0D822C75-7240-42E3-8780-18CABC9C45A1@netasq.com> <20120903212340.GE88081@ithaqua.etoilebsd.net> <60A32BCE-756A-4A0D-A24D-7AC42F71BD1A@netasq.com> <20120904192320.GF88081@ithaqua.etoilebsd.net> To: Baptiste Daroussin X-Mailer: Apple Mail (2.1278) Cc: freebsd-current Subject: Re: [CFT] hwpmc support for Intel Ivy Bridge X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 07:33:56 -0000 >=20 > Hi, >=20 > here are the results >=20 > # pmccontrol -L > SOFT > CLOCK.STAT > CLOCK.HARD > LOCK.FAILED > PAGE_FAULT.WRITE > PAGE_FAULT.READ > PAGE_FAULT.ALL >=20 Seems that the CPU was not detected can you dump the dmesg with CPU: = section ?=20 As Davide ask, if you are in a VM pmc are not usable in the virtualized = HW. > # pmcstat -SOFF_CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY -w1 -T > pmcstat: ERROR: Cannot allocate system-mode pmc with specification = "OFF_CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY": Invalid argument >=20 > # pmcstat -n2000000 -S INSTR_RETIRED_ANY -w4 -T > pmcstat: ERROR: Cannot allocate system-mode pmc with specification = "INSTR_RETIRED_ANY": Invalid argument >=20 > # pmcstat -n2000000 -S INSTR_RETIRED_ANY -w4 -T > pmcstat: ERROR: Cannot allocate system-mode pmc with specification = "INSTR_RETIRED_ANY": Invalid argument >=20 > # pmcstat -SPAGE_FAULT.ALL -w4 -T > (This one seems to work correctly >=20 > # pmcstat -SOFF_CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY = -n2000000 -SCPU_CLK_UNHALTED.THREAD_P -n2000000 -S INSTR_RETIRED_ANY = -SPAGE_FAULT > pmcstat: ERROR: Cannot allocate system-mode pmc with specification = "OFF_CORE_RESPONSE_0,rsp=3DREQ_DMND_DATA_RD+RES_ANY": Invalid argument >=20 > The pmctest.py fails:=20 > # ./test/hwpmc/pmctest.py > Traceback (most recent call last): > File "./test/hwpmc/pmctest.py", line 94, in > main() > File "./test/hwpmc/pmctest.py", line 81, in main > stdout=3DPIPE) > File "/usr/local/lib/python2.7/subprocess.py", line 679, in __init__ > errread, errwrite) > File "/usr/local/lib/python2.7/subprocess.py", line 1249, in = _execute_child > raise child_exception > TypeError: execv() arg 2 must contain only strings >=20 >=20 > regards, > Bapt From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 08:03:06 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 76B671065678 for ; Wed, 5 Sep 2012 08:03:06 +0000 (UTC) (envelope-from bapt@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3E86C8FC1A; Wed, 5 Sep 2012 08:03:06 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q858365V089785; Wed, 5 Sep 2012 08:03:06 GMT (envelope-from bapt@FreeBSD.org) Received: (from bapt@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q85835ej089762; Wed, 5 Sep 2012 08:03:05 GMT (envelope-from bapt@FreeBSD.org) X-Authentication-Warning: freefall.freebsd.org: bapt set sender to bapt@FreeBSD.org using -f Date: Wed, 5 Sep 2012 10:03:01 +0200 From: Baptiste Daroussin To: Fabien Thomas Message-ID: <20120905080301.GJ88081@ithaqua.etoilebsd.net> References: <0D822C75-7240-42E3-8780-18CABC9C45A1@netasq.com> <20120903212340.GE88081@ithaqua.etoilebsd.net> <60A32BCE-756A-4A0D-A24D-7AC42F71BD1A@netasq.com> <20120904192320.GF88081@ithaqua.etoilebsd.net> <441C9DEC-517B-4991-A2F9-33E8080C0B19@netasq.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pWOmaDnDlrCGjNh4" Content-Disposition: inline In-Reply-To: <441C9DEC-517B-4991-A2F9-33E8080C0B19@netasq.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-current Subject: Re: [CFT] hwpmc support for Intel Ivy Bridge X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 08:03:06 -0000 --pWOmaDnDlrCGjNh4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Sep 05, 2012 at 09:33:55AM +0200, Fabien Thomas wrote: > >=20 > > Hi, > >=20 > > here are the results > >=20 > > # pmccontrol -L > > SOFT > > CLOCK.STAT > > CLOCK.HARD > > LOCK.FAILED > > PAGE_FAULT.WRITE > > PAGE_FAULT.READ > > PAGE_FAULT.ALL > >=20 > Seems that the CPU was not detected can you dump the dmesg with CPU: sect= ion ?=20 > As Davide ask, if you are in a VM pmc are not usable in the virtualized H= W. >=20 I'm on real hardware: CPU: Intel(R) Core(TM) i5-3317U CPU @ 1.70GHz (1696.19-MHz K8-class CPU) Origin =3D "GenuineIntel" Id =3D 0x306a9 Family =3D 6 Model =3D 3a St= epping =3D 9 Features=3D0xbfebfbff Features2=3D0x7fbae3bf AMD Features=3D0x28100800 AMD Features2=3D0x1 TSC: P-state invariant, performance statistics regards, Bapt --pWOmaDnDlrCGjNh4 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlBHBzUACgkQ8kTtMUmk6EzZaACgn1eTOYeF6jGvPQHidDCuqf1f uyYAn3+Kt9j0SmAfK0L4X+F3k26z/Z2k =DFnk -----END PGP SIGNATURE----- --pWOmaDnDlrCGjNh4-- From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 08:11:45 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 29103106564A for ; Wed, 5 Sep 2012 08:11:45 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id D2E748FC15 for ; Wed, 5 Sep 2012 08:11:44 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.69) for freebsd-current@FreeBSD.org with esmtp (envelope-from ) id <1T9Ai7-0002gz-Qp>; Wed, 05 Sep 2012 10:11:43 +0200 Received: from e178008184.adsl.alicedsl.de ([85.178.8.184] helo=thor.walstatt.dyndns.org) by inpost2.zedat.fu-berlin.de (Exim 4.69) for freebsd-current@FreeBSD.org with esmtpsa (envelope-from ) id <1T9Ai7-0003R8-LR>; Wed, 05 Sep 2012 10:11:43 +0200 Message-ID: <5047093A.5060408@zedat.fu-berlin.de> Date: Wed, 05 Sep 2012 10:11:38 +0200 From: "O. Hartmann" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120810 Thunderbird/14.0 MIME-Version: 1.0 To: Current FreeBSD X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig18D5BDD23CD69B7671F24BEB" X-Originating-IP: 85.178.8.184 Cc: Subject: Firefox-15/Thunderbird-15: won't compile on FreeBSD 10.0-CUR: /jsproxy.h:17:7: error: visibility does not match previous declaration X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 08:11:45 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig18D5BDD23CD69B7671F24BEB Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello. Udating/reinstalling of both ports www/firefox (15.0) and mail/thunderbird (15) fail with an error like showed below. Last time I saw this on FreeBSD 10.0-CUR ( r240108M), it was almost the same issue due to the compiler change from CLANG 3.0 -> 3.1 as far as I experienced and has been fixed later. Just wondering if this is a regression ... Oliver clang++ -o jsapi.o -c -I./../../dist/system_wrappers_js -include =2E/config/gcc_hidden.h -DENABLE_YARR_JIT=3D1 -DMOZ_GLUE_IN_PROGRAM -DEXPORT_JS_API -DJS_HAS_CTYPES -DDLL_PREFIX=3D\"lib\" -DDLL_SUFFIX=3D\".so\" -DNO_NSPR_10_SUPPORT -I/usr/local/lib/libffi-3.0.9/include -I. -I./../../mfbt/double-conversion -I. -I. -I./../../dist/include -I./../../dist/include/nsprpub -I/usr/local/include/nspr -I. -I./assembler -I./yarr -fPIC -Qunused-arguments -isystem/usr/local/include -I/usr/local/include/nss -I/usr/local/include/nss/nss -I/usr/local/include -fno-rtti -Qunused-arguments -Wall -Wpointer-arith -Woverloaded-virtual -Werror=3Dreturn-type -Wtype-limits -Wempty-body -Wno-ctor-dtor-privacy -Wno-overlength-strings -Wno-invalid-offsetof -Wno-variadic-macros -Wno-c++0x-extensions -Wno-extended-offsetof -Wno-unknown-warning-option -Wno-return-type-c-linkage -O2 -fno-strict-aliasing -pipe -pipe -O3 -fno-strict-aliasing -march=3Dnative -ffunction-sections -fdata-sections -pipe -DNDEBUG -DTRIMMED -O2 -O3 -fomit-frame-pointer -DUSE_SYSTEM_MALLOC=3D1 -DENABLE_ASSEMBLER=3D1 -DENABLE_JIT=3D1 -Qunused-arguments -isystem/usr/local/include -I/usr/local/include/nss -I/usr/local/include/nss/nss -I/usr/local/include -DMOZILLA_CLIENT -include ./js-confdefs.h -MD -MF =2Edeps/jsapi.pp /usr/ports/www/firefox/work/mozilla-release/js/src/jsapi= =2Ecpp In file included from /usr/ports/www/firefox/work/mozilla-release/js/src/jsanalyze.cpp:12: In file included from ./jsinferinlines.h:18: In file included from ./vm/Stack-inl.h:17: In file included from ./jsscriptinlines.h:22: In file included from ./jsscopeinlines.h:28: In file included from ./jsobjinlines.h:24: =2E/jsproxy.h:17:7: error: visibility does not match previous declaration= class JS_FRIEND_API(BaseProxyHandler) { --------------enig18D5BDD23CD69B7671F24BEB Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBAgAGBQJQRwk/AAoJEOgBcD7A/5N8cv4IANKCA4iVx6do4hrXAFbhyy+L N4EpfBhz0e/z332ShNtBXnZ4gxGFVfL83FL5kK0XHkpv9i+sEARiducLCbrkn0ab Qixs1x/k2hmDH4n2ySf6FnNP7J9pYm1hRgFBv9PzCngFEhcuVU3Cwv2WPblBgJ0z UgA8Q7b3J5cyKtigwZAlsh4w10xSsGO9M6fa8RMRwpmZj893df9F2Xq8qzs+1UTa Fst7ugkp69KRgNC8WmnCov9SZ3YK/XSkmPCx6neH0t8hPAKfCHZYo3wgJGqCEhfr HhYOibEuGQn3wMaMHMtOMyr1icHzcMPG3fKynQc62cImzT54YctsD/ihzm0R2lw= =V6IG -----END PGP SIGNATURE----- --------------enig18D5BDD23CD69B7671F24BEB-- From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 09:11:53 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3B2F6106566B for ; Wed, 5 Sep 2012 09:11:53 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id E97D38FC0C for ; Wed, 5 Sep 2012 09:11:52 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:d43c:3039:49b5:2229] (unknown [IPv6:2001:7b8:3a7:0:d43c:3039:49b5:2229]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 4D9505C44; Wed, 5 Sep 2012 11:11:51 +0200 (CEST) Message-ID: <50471757.4070202@FreeBSD.org> Date: Wed, 05 Sep 2012 11:11:51 +0200 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20120828 Thunderbird/16.0 MIME-Version: 1.0 To: "O. Hartmann" References: <5047093A.5060408@zedat.fu-berlin.de> In-Reply-To: <5047093A.5060408@zedat.fu-berlin.de> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Current FreeBSD Subject: Re: Firefox-15/Thunderbird-15: won't compile on FreeBSD 10.0-CUR: /jsproxy.h:17:7: error: visibility does not match previous declaration X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 09:11:53 -0000 On 2012-09-05 10:11, O. Hartmann wrote: > Udating/reinstalling of both ports www/firefox (15.0) and > mail/thunderbird (15) fail with an error like showed below. .... > ./jsproxy.h:17:7: error: visibility does not match previous declaration > class JS_FRIEND_API(BaseProxyHandler) { Please see: http://llvm.org/bugs/show_bug.cgi?id=13688 This is either a bug in clang 3.2, or alternatively, a problem with inconsistent declarations in libc++. From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 09:31:25 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 397FA106564A for ; Wed, 5 Sep 2012 09:31:25 +0000 (UTC) (envelope-from dimitry@andric.com) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) by mx1.freebsd.org (Postfix) with ESMTP id E44A68FC12 for ; Wed, 5 Sep 2012 09:31:24 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:d43c:3039:49b5:2229] (unknown [IPv6:2001:7b8:3a7:0:d43c:3039:49b5:2229]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id A06145C44; Wed, 5 Sep 2012 11:31:23 +0200 (CEST) Message-ID: <50471BEE.6030708@andric.com> Date: Wed, 05 Sep 2012 11:31:26 +0200 From: Dimitry Andric User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20120828 Thunderbird/16.0 MIME-Version: 1.0 To: Garrett Cooper References: <5046670C.6050500@andric.com> <20120904214344.GA17723@troutmask.apl.washington.edu> <504679CB.90204@andric.com> <20120904221413.GA19395@troutmask.apl.washington.edu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, Steve Kargl Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 09:31:25 -0000 On 2012-09-05 01:40, Garrett Cooper wrote: ... > Steve does have a point. Posting the results of > CFLAGS/CPPFLAGS/LDFLAGS/etc for config.log (and maybe poking through > the code to figure out what *FLAGS were used elsewhere) is more > valuable than the data is in its current state (unfortunately.. > autoconf makes things more complicated). Just to note, autoconf is not used in the FreeBSD source tree, so it does not apply to the first two builds in the performance test (e.g. building in-tree clang and gcc). The other build is Boost, which has yet another totally different build system, based on Perforce's Jam. Again, no autoconf. In any case, for all three builds, the default optimization options were used. Basically: 1) For building the FreeBSD in-tree version of clang 3.2: -O2 -pipe -fno-strict-aliasing These are just the default FreeBSD optimization flags for building clang, which are probably used by the majority of users out there. This is the case that I was interested in particularly. The -fno-strict-aliasing is not really my choice, but it was introduced in the past by Nathan Whitehorn, who apparently saw problems without it. It will hopefully disappear in the future. 2) For building the FreeBSD in-tree version of gcc 4.2.1: -O2 -pipe These are the default FreeBSD optimization flags. 3) For building Boost 1.50.0: -ftemplate-depth-128 -O3 -finline-functions These are the Boost defaults for gcc-compatible compilers, from tools/build/v2/tools/gcc.jam. From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 09:37:03 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE99A106566C for ; Wed, 5 Sep 2012 09:37:03 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from theravensnest.org (theraven.freebsd.your.org [216.14.102.27]) by mx1.freebsd.org (Postfix) with ESMTP id 782D78FC0A for ; Wed, 5 Sep 2012 09:37:03 +0000 (UTC) Received: from c120.sec.cl.cam.ac.uk (c120.sec.cl.cam.ac.uk [128.232.18.120]) (authenticated bits=0) by theravensnest.org (8.14.5/8.14.5) with ESMTP id q859apsm089083 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Wed, 5 Sep 2012 09:36:52 GMT (envelope-from theraven@FreeBSD.org) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=iso-8859-1 From: David Chisnall In-Reply-To: <50471BEE.6030708@andric.com> Date: Wed, 5 Sep 2012 10:36:50 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <96BD00DE-865C-4690-A2F1-E5B7C5D221C0@FreeBSD.org> References: <5046670C.6050500@andric.com> <20120904214344.GA17723@troutmask.apl.washington.edu> <504679CB.90204@andric.com> <20120904221413.GA19395@troutmask.apl.washington.edu> <50471BEE.6030708@andric.com> To: Dimitry Andric X-Mailer: Apple Mail (2.1278) Cc: Garrett Cooper , freebsd-current@FreeBSD.org, Steve Kargl Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 09:37:03 -0000 On 5 Sep 2012, at 10:31, Dimitry Andric wrote: > These are just the default FreeBSD optimization flags for building > clang, which are probably used by the majority of users out there. > This is the case that I was interested in particularly. The > -fno-strict-aliasing is not really my choice, but it was introduced > in the past by Nathan Whitehorn, who apparently saw problems without > it. It will hopefully disappear in the future. Clang currently defaults to no strict aliasing on FreeBSD. In my = experience, most C programmers misunderstand the aliasing rules of C and = even people on the C++ standards committee often get them wrong for C++, = so trading a 1-10% performance increase for a significant chance of = generating non-working code seems like a poor gain. If people are = certain that they do understand the rules, then they can add = -fstrict-aliasing to their own CFLAGS. David= From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 10:56:06 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6469106566C; Wed, 5 Sep 2012 10:56:06 +0000 (UTC) (envelope-from dimitry@andric.com) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id 8D5748FC0A; Wed, 5 Sep 2012 10:56:06 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:d43c:3039:49b5:2229] (unknown [IPv6:2001:7b8:3a7:0:d43c:3039:49b5:2229]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id B89355C43; Wed, 5 Sep 2012 12:56:05 +0200 (CEST) Message-ID: <50472FC7.8010500@andric.com> Date: Wed, 05 Sep 2012 12:56:07 +0200 From: Dimitry Andric User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20120828 Thunderbird/16.0 MIME-Version: 1.0 To: David Chisnall References: <5046670C.6050500@andric.com> <20120904214344.GA17723@troutmask.apl.washington.edu> <504679CB.90204@andric.com> <20120904221413.GA19395@troutmask.apl.washington.edu> <50471BEE.6030708@andric.com> <96BD00DE-865C-4690-A2F1-E5B7C5D221C0@FreeBSD.org> In-Reply-To: <96BD00DE-865C-4690-A2F1-E5B7C5D221C0@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Garrett Cooper , freebsd-current@FreeBSD.org, Steve Kargl Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 10:56:06 -0000 On 2012-09-05 11:36, David Chisnall wrote: > On 5 Sep 2012, at 10:31, Dimitry Andric wrote: >> TThe >> -fno-strict-aliasing is not really my choice, but it was introduced >> in the past by Nathan Whitehorn, who apparently saw problems without >> it. It will hopefully disappear in the future. > Clang currently defaults to no strict aliasing on FreeBSD. Yes, but upstream has never used -fno-strict-aliasing, just plain -O2. I run regular separate builds of pristine upstream clang on FreeBSD, and I haven't seen any failures due aliasing problems in all the regression tests. That doesn't guarantee there are no problems, of course... > In my experience, most C programmers misunderstand the aliasing rules of C and even people on the C++ standards committee often get them wrong for C++, so trading a 1-10% performance increase for a significant chance of generating non-working code seems like a poor gain. If people are certain that they do understand the rules, then they can add -fstrict-aliasing to their own CFLAGS. I'm actually quite interested in the performance difference; I think I will run a few tests. :) From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 13:11:25 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0970C106564A; Wed, 5 Sep 2012 13:11:25 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 868358FC14; Wed, 5 Sep 2012 13:11:24 +0000 (UTC) Received: by vcbgb30 with SMTP id gb30so949268vcb.13 for ; Wed, 05 Sep 2012 06:11:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=G8nGq03LshHGsGSBGdoCQ72669Xy6L7ngfRKazblHxI=; b=KHoYMVkfgmbtqokINGuuYyOmwrGm/vE3DBR6sGHB9TtPI29cHRb1TQMD0M1v4QDIAD dQ+FiNaMErkgSKFEiX6pLI6kcem/Iz2BnOB75fMxQ1G6mHLMqj1KRCJHq/GvF035Hkbz 5i1eZMB/1CH20A/q2g8sOFsbR08PqnwI55LPcvaLTiXKZg/ymux0T7+IXgFc22LTo5Mz 3DvJZh1bGrHnPZFzedYc1PI8MS5mZ/JENHoabPs/2AQQU3ukBlInaRv5CoAEpOVRam3S 4AvAlAEQGptvg04imNmCIoAje269xciNg3mtVPak2migtPcdLqRNW+T0p18gwNqYtow/ b2yA== MIME-Version: 1.0 Received: by 10.220.223.5 with SMTP id ii5mr2454194vcb.51.1346850682664; Wed, 05 Sep 2012 06:11:22 -0700 (PDT) Received: by 10.58.203.136 with HTTP; Wed, 5 Sep 2012 06:11:22 -0700 (PDT) In-Reply-To: <50472FC7.8010500@andric.com> References: <5046670C.6050500@andric.com> <20120904214344.GA17723@troutmask.apl.washington.edu> <504679CB.90204@andric.com> <20120904221413.GA19395@troutmask.apl.washington.edu> <50471BEE.6030708@andric.com> <96BD00DE-865C-4690-A2F1-E5B7C5D221C0@FreeBSD.org> <50472FC7.8010500@andric.com> Date: Wed, 5 Sep 2012 09:11:22 -0400 Message-ID: From: Justin Hibbits To: Dimitry Andric Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Garrett Cooper , freebsd-current@freebsd.org, David Chisnall , Steve Kargl Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 13:11:25 -0000 On Wed, Sep 5, 2012 at 6:56 AM, Dimitry Andric wrote: > On 2012-09-05 11:36, David Chisnall wrote: > >> On 5 Sep 2012, at 10:31, Dimitry Andric wrote: >> >>> TThe >>> >>> -fno-strict-aliasing is not really my choice, but it was introduced >>> in the past by Nathan Whitehorn, who apparently saw problems without >>> it. It will hopefully disappear in the future. >>> >> Clang currently defaults to no strict aliasing on FreeBSD. >> > > Yes, but upstream has never used -fno-strict-aliasing, just plain -O2. > I run regular separate builds of pristine upstream clang on FreeBSD, and > I haven't seen any failures due aliasing problems in all the regression > tests. That doesn't guarantee there are no problems, of course... Aliasing problems are seen much more frequently on PowerPC than any other platform for Clang. I found this a while back when doing some Clang testing, and I still see problems with upstream unless I explicitly set -fno-strict-aliasing. Nathan had mentioned wanting to get upstream to use -fno-strict-aliasing by default on all platforms, but I don't think that ever made it beyond his suggesting. I filed this bug to track it: http://llvm.org/bugs/show_bug.cgi?id=11955 In my experience, most C programmers misunderstand the aliasing rules of C >> and even people on the C++ standards committee often get them wrong for >> C++, so trading a 1-10% performance increase for a significant chance of >> generating non-working code seems like a poor gain. If people are certain >> that they do understand the rules, then they can add -fstrict-aliasing to >> their own CFLAGS. >> > > I'm actually quite interested in the performance difference; I think I > will run a few tests. :) From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 13:27:31 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CBF971065689; Wed, 5 Sep 2012 13:27:31 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (m209-73.dsl.rawbw.com [198.144.209.73]) by mx1.freebsd.org (Postfix) with ESMTP id 9CAA18FC15; Wed, 5 Sep 2012 13:27:31 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.5/8.14.5) with ESMTP id q85DRP3i006498; Wed, 5 Sep 2012 06:27:25 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.5/8.14.5/Submit) id q85DRP06006497; Wed, 5 Sep 2012 06:27:25 -0700 (PDT) (envelope-from david) Date: Wed, 5 Sep 2012 06:27:25 -0700 From: David Wolfskill To: current@freebsd.org Message-ID: <20120905132725.GE1486@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="gMR3gsNFwZpnI/Ts" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Cc: delphij@freebsd.org Subject: buildkernel fails after r240104 (i386, clang) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 13:27:31 -0000 --gMR3gsNFwZpnI/Ts Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Running: FreeBSD g1-227.catwhisker.org 10.0-CURRENT FreeBSD 10.0-CURRENT #669 240081= M: Tue Sep 4 05:02:11 PDT 2012 root@g1-227.catwhisker.org:/usr/obj/usr= /src/sys/CANARY i386 After updating sources to r240131, I see: =2E.. >>> stage 3.2: building everything =2E.. clang -c -O -pipe -std=3Dc99 -g -Wall -Wredundant-decls -Wnested-externs -= Wstrict -prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wu= ndef=20 -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics= -show -option -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-= paren theses-equality -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -= D_KER NEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -mno-aes -mno-avx -= mno-m mx -mno-sse -msoft-float -ffreestanding -fstack-protector -Werror /usr/src= /sys/ dev/fe/if_fe_isa.c ctfconvert -L VERSION -g if_ed_wd80x3.o clang -c -O -pipe -std=3Dc99 -g -Wall -Wredundant-decls -Wnested-externs -= Wstrict -prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wu= ndef=20 -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics= -show -option -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-= paren theses-equality -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -= D_KER NEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -mno-aes -mno-avx -= mno-m mx -mno-sse -msoft-float -ffreestanding -fstack-protector -Werror /usr/src= /sys/ dev/hpt27xx/osm_bsd.c clang -c -O -pipe -std=3Dc99 -g -Wall -Wredundant-decls -Wnested-externs -= Wstrict -prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wu= ndef=20 -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics= -show -option -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-= paren theses-equality -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -= D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -mno-aes -mno-= avx -mno-mmx -mno-sse -msoft-float -ffreestanding -fstack-protector -Werror= /usr/src/sys/dev/hpt27xx/os_bsd.c /usr/src/sys/dev/hpt27xx/osm_bsd.c:170:66: warning: for loop has empty body= [-Wempty-body] for (order=3D0, size=3DPAGE_SIZE; sizesize; order++, si= ze<<=3D1) ; = ^ /usr/src/sys/dev/hpt27xx/osm_bsd.c:170:66: note: put the semicolon on a sep= arate line to silence this warning /usr/src/sys/dev/hpt27xx/osm_bsd.c:1180:25: error: format string is not a s= tring literal (potentially insecure) [-Werror,-Wformat-security] S_IRUSR | S_IWUSR, driver_name); ^~~~~~~~~~~ /usr/src/sys/dev/hpt27xx/hpt27xx_config.h:46:21: note: expanded from macro = 'driver_name' #define driver_name hpt27xx_driver_name ^~~~~~~~~~~~~~~~~~~ 1 warning and 1 error generated. *** [osm_bsd.o] Error code 1 ctfconvert -L VERSION -g if_fe_isa.o ctfconvert -L VERSION -g os_bsd.o 1 warning generated. ctfconvert -L VERSION -g if_bwn.o ld -d -warn-common -r -d -o if_bwn.kld if_bwn.o ctfmerge -L VERSION -g -o if_bwn.kld if_bwn.o :> export_syms awk -f /usr/src/sys/conf/kmod_syms.awk if_bwn.kld export_syms | xargs -J% = objcopy % if_bwn.kld ld -Bshareable -d -warn-common -o if_bwn.ko.debug if_bwn.kld objcopy --only-keep-debug if_bwn.ko.debug if_bwn.ko.symbols objcopy --strip-debug --add-gnu-debuglink=3Dif_bwn.ko.symbols if_bwn.ko.deb= ug if_bwn.ko =2E.. =3D=3D=3D> zlib (all) 1 error *** [buildkernel] Error code 2 1 error *** [buildkernel] Error code 2 1 error I suspect that: ------------------------------------------------------------------------ r240104 | delphij | 2012-09-04 14:02:57 -0700 (Tue, 04 Sep 2012) | 4 lines Add hpt27xx to GENERIC kernel for amd64 and i386 systems. MFC after: 2 weeks ------------------------------------------------------------------------ (from "svn log sys/conf/i386/GENERIC") has a fair bit of involvement. Peace, david --=20 David H. Wolfskill david@catwhisker.org Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --gMR3gsNFwZpnI/Ts Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlBHUzwACgkQmprOCmdXAD1+dQCgh+hSa9DSj8MVFfAoOJmzlQZZ DeEAnjFwM2878rQip0hEksaqlsMIfKGB =wSnk -----END PGP SIGNATURE----- --gMR3gsNFwZpnI/Ts-- From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 14:11:22 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2244A106564A for ; Wed, 5 Sep 2012 14:11:22 +0000 (UTC) (envelope-from ohartman@mail.zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id CC5218FC08 for ; Wed, 5 Sep 2012 14:11:21 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.69) for freebsd-current@freebsd.org with esmtp (envelope-from ) id <1T9GK8-0002KG-NM>; Wed, 05 Sep 2012 16:11:20 +0200 Received: from telesto.geoinf.fu-berlin.de ([130.133.86.198]) by inpost2.zedat.fu-berlin.de (Exim 4.69) for freebsd-current@freebsd.org with esmtpsa (envelope-from ) id <1T9GK8-0005Nv-Jp>; Wed, 05 Sep 2012 16:11:20 +0200 Message-ID: <50475D83.6090803@mail.zedat.fu-berlin.de> Date: Wed, 05 Sep 2012 16:11:15 +0200 From: "O. Hartmann" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120810 Thunderbird/14.0 MIME-Version: 1.0 To: Current FreeBSD X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig27092B67FF4A51E57AD102B0" X-Originating-IP: 130.133.86.198 X-Mailman-Approved-At: Wed, 05 Sep 2012 15:23:03 +0000 Subject: atomic_ops.h: missing ... X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 14:11:22 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig27092B67FF4A51E57AD102B0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello. While fiddling around with software that is looking for an include file "atomic_ops.h", which seems to reside in the FreeBSD operating system's sources with lib/lbkse, I'd like to know whether those architecture specific header files are installed in some places, where they could be found by the regular ports building environment (without necessarily having the OS sources installed). I'm working on FreeBSD 10.0-CUR with OS sources installed. Doing a "locate atomic_ops.h" reveals /usr/local/include/cpl_atomic_ops.h /usr/src/lib/libkse/arch/amd64/include/atomic_ops.h /usr/src/lib/libkse/arch/arm/include/atomic_ops.h /usr/src/lib/libkse/arch/i386/include/atomic_ops.h /usr/src/lib/libkse/arch/ia64/include/atomic_ops.h /usr/src/lib/libkse/arch/powerpc/include/atomic_ops.h /usr/src/lib/libkse/arch/sparc64/include/atomic_ops.h Is this include missed by intention or is it a bug? Thanks in advance. Oliver --------------enig27092B67FF4A51E57AD102B0 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBAgAGBQJQR12IAAoJEOgBcD7A/5N8BHAH/0mwThQryhBrq+K+fVTK4C2z iUsRGgvALvJXAxj4vZ0Jn1xzt8IVCPhhwjxGvrfhfYz3UAjR4XpG5C1VoQZTKZnm yluDM+URgLLVzNKRD14x1dC6bU0Kqio8o1Woqevk3jNbds0e3GzHf2AV2n3iCsVn k3kvrT+kn12lTwuXD1JVVfGwbh4A2W4c3qXo53kMaXm7ihmmq8el8TzFLGDIMbSu c9jnEMES5lrPmyG5gL63SoPic/nChBNqthbmcIsWXiKSKT1Dvp7SpU700ELY/+gF 4QhXRB/fwn9UK2GnjYX87QwhQPu0QXHj4LNSgt+IqwHd3inHJqYEtBvl3ePEfZw= =qSzx -----END PGP SIGNATURE----- --------------enig27092B67FF4A51E57AD102B0-- From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 15:40:39 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6654106566C for ; Wed, 5 Sep 2012 15:40:39 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 44E248FC15 for ; Wed, 5 Sep 2012 15:40:38 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q85FecHL079328 for ; Wed, 5 Sep 2012 09:40:38 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q85FeFLT042731; Wed, 5 Sep 2012 09:40:15 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: "O. Hartmann" In-Reply-To: <50475D83.6090803@mail.zedat.fu-berlin.de> References: <50475D83.6090803@mail.zedat.fu-berlin.de> Content-Type: text/plain; charset="us-ascii" Date: Wed, 05 Sep 2012 09:40:15 -0600 Message-ID: <1346859615.59094.79.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Current FreeBSD Subject: Re: atomic_ops.h: missing ... X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 15:40:40 -0000 On Wed, 2012-09-05 at 16:11 +0200, O. Hartmann wrote: > Hello. > > While fiddling around with software that is looking for an include file > "atomic_ops.h", which seems to reside in the FreeBSD operating system's > sources with lib/lbkse, I'd like to know whether those architecture > specific header files are installed in some places, where they could be > found by the regular ports building environment (without necessarily > having the OS sources installed). > > I'm working on FreeBSD 10.0-CUR with OS sources installed. > > Doing a "locate atomic_ops.h" reveals > > /usr/local/include/cpl_atomic_ops.h > /usr/src/lib/libkse/arch/amd64/include/atomic_ops.h > /usr/src/lib/libkse/arch/arm/include/atomic_ops.h > /usr/src/lib/libkse/arch/i386/include/atomic_ops.h > /usr/src/lib/libkse/arch/ia64/include/atomic_ops.h > /usr/src/lib/libkse/arch/powerpc/include/atomic_ops.h > /usr/src/lib/libkse/arch/sparc64/include/atomic_ops.h > > Is this include missed by intention or is it a bug? > > Thanks in advance. There also used to be an atomic_ops.h in the libpthread implementation in days of old. I think both it and the one in libkse are intended to be private to the library implementation and they don't get installed. -- Ian From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 15:46:59 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 99D081065670 for ; Wed, 5 Sep 2012 15:46:59 +0000 (UTC) (envelope-from tijl@coosemans.org) Received: from mailrelay003.isp.belgacom.be (mailrelay003.isp.belgacom.be [195.238.6.53]) by mx1.freebsd.org (Postfix) with ESMTP id 1D0338FC18 for ; Wed, 5 Sep 2012 15:46:58 +0000 (UTC) X-Belgacom-Dynamic: yes X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av8EABdzR1BbsRjw/2dsb2JhbABFt2aDPYEIgiABAQVWIxALEgYJJQ8CKBAOBg0BBwEBiA26c4sRhwIDjmKBII8KhmeCZQ Received: from 240.24-177-91.adsl-dyn.isp.belgacom.be (HELO kalimero.tijl.coosemans.org) ([91.177.24.240]) by relay.skynet.be with ESMTP; 05 Sep 2012 17:46:57 +0200 Received: from kalimero.tijl.coosemans.org (kalimero.tijl.coosemans.org [127.0.0.1]) by kalimero.tijl.coosemans.org (8.14.5/8.14.5) with ESMTP id q85Fktm3015572; Wed, 5 Sep 2012 17:46:56 +0200 (CEST) (envelope-from tijl@coosemans.org) Message-ID: <504773EA.40806@coosemans.org> Date: Wed, 05 Sep 2012 17:46:50 +0200 From: Tijl Coosemans User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:14.0) Gecko/20120804 Thunderbird/14.0 MIME-Version: 1.0 To: "O. Hartmann" References: <50475D83.6090803@mail.zedat.fu-berlin.de> In-Reply-To: <50475D83.6090803@mail.zedat.fu-berlin.de> X-Enigmail-Version: 1.4.2 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------enigE3C8DDC8BA5BCA5ABC888B28" Cc: Current FreeBSD Subject: Re: atomic_ops.h: missing ... X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 15:46:59 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigE3C8DDC8BA5BCA5ABC888B28 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 05-09-2012 16:11, O. Hartmann wrote: > Hello. >=20 > While fiddling around with software that is looking for an include file= > "atomic_ops.h", which seems to reside in the FreeBSD operating system's= > sources with lib/lbkse, I'd like to know whether those architecture > specific header files are installed in some places, where they could be= > found by the regular ports building environment (without necessarily > having the OS sources installed). >=20 > I'm working on FreeBSD 10.0-CUR with OS sources installed. >=20 > Doing a "locate atomic_ops.h" reveals >=20 > /usr/local/include/cpl_atomic_ops.h > /usr/src/lib/libkse/arch/amd64/include/atomic_ops.h > /usr/src/lib/libkse/arch/arm/include/atomic_ops.h > /usr/src/lib/libkse/arch/i386/include/atomic_ops.h > /usr/src/lib/libkse/arch/ia64/include/atomic_ops.h > /usr/src/lib/libkse/arch/powerpc/include/atomic_ops.h > /usr/src/lib/libkse/arch/sparc64/include/atomic_ops.h >=20 > Is this include missed by intention or is it a bug? It could be looking for the header from this port: devel/libatomic_ops --------------enigE3C8DDC8BA5BCA5ABC888B28 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iF4EAREIAAYFAlBHc+8ACgkQfoCS2CCgtitluQD/WI0CXY36Gx/mbZJn6nMBJsgu wXUV6LpzTsxzQfHibdgA/jam87DSoRS9NvmvXL4Ic++DkLJQqi0n/CyvCjmqeblN =A6X2 -----END PGP SIGNATURE----- --------------enigE3C8DDC8BA5BCA5ABC888B28-- From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 17:13:31 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA8AF106564A; Wed, 5 Sep 2012 17:13:31 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id 767CE8FC0A; Wed, 5 Sep 2012 17:13:31 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:d43c:3039:49b5:2229] (unknown [IPv6:2001:7b8:3a7:0:d43c:3039:49b5:2229]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id EFFBC5C43; Wed, 5 Sep 2012 19:13:29 +0200 (CEST) Message-ID: <5047883A.9060905@FreeBSD.org> Date: Wed, 05 Sep 2012 19:13:30 +0200 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20120828 Thunderbird/16.0 MIME-Version: 1.0 To: David Wolfskill References: <20120905132725.GE1486@albert.catwhisker.org> In-Reply-To: <20120905132725.GE1486@albert.catwhisker.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: delphij@freebsd.org, current@freebsd.org Subject: Re: buildkernel fails after r240104 (i386, clang) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 17:13:31 -0000 On 2012-09-05 15:27, David Wolfskill wrote: ... > /usr/src/sys/dev/hpt27xx/osm_bsd.c:1180:25: error: format string is not a string literal (potentially insecure) [-Werror,-Wformat-security] > S_IRUSR | S_IWUSR, driver_name); > ^~~~~~~~~~~ Thanks, fixed in r240144. From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 17:37:13 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 74AEA106566C; Wed, 5 Sep 2012 17:37:13 +0000 (UTC) (envelope-from rdivacky@vlakno.cz) Received: from vlakno.cz (vlakno.cz [46.28.110.116]) by mx1.freebsd.org (Postfix) with ESMTP id 2EF018FC12; Wed, 5 Sep 2012 17:37:13 +0000 (UTC) Received: by vlakno.cz (Postfix, from userid 1002) id 93F7F7F3888; Wed, 5 Sep 2012 19:37:04 +0200 (CEST) Date: Wed, 5 Sep 2012 19:37:04 +0200 From: Roman Divacky To: Justin Hibbits Message-ID: <20120905173704.GA31250@freebsd.org> References: <5046670C.6050500@andric.com> <20120904214344.GA17723@troutmask.apl.washington.edu> <504679CB.90204@andric.com> <20120904221413.GA19395@troutmask.apl.washington.edu> <50471BEE.6030708@andric.com> <96BD00DE-865C-4690-A2F1-E5B7C5D221C0@FreeBSD.org> <50472FC7.8010500@andric.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Garrett Cooper , Dimitry Andric , freebsd-current@freebsd.org, David Chisnall , Steve Kargl Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 17:37:13 -0000 What makes you think it's a bug in llvm code and not a plain gcc miscompile? Other people seem to compile llvm on PPC64 with gcc and -fstrict-aliasing just fine. They just dont happen to use gcc4.2.1. Ie. gcc47 is reported to not have this problem. I personally can confirm that fbsd+gcc48 is ok to On Wed, Sep 05, 2012 at 09:11:22AM -0400, Justin Hibbits wrote: > On Wed, Sep 5, 2012 at 6:56 AM, Dimitry Andric wrote: > > > On 2012-09-05 11:36, David Chisnall wrote: > > > >> On 5 Sep 2012, at 10:31, Dimitry Andric wrote: > >> > >>> TThe > >>> > >>> -fno-strict-aliasing is not really my choice, but it was introduced > >>> in the past by Nathan Whitehorn, who apparently saw problems without > >>> it. It will hopefully disappear in the future. > >>> > >> Clang currently defaults to no strict aliasing on FreeBSD. > >> > > > > Yes, but upstream has never used -fno-strict-aliasing, just plain -O2. > > I run regular separate builds of pristine upstream clang on FreeBSD, and > > I haven't seen any failures due aliasing problems in all the regression > > tests. That doesn't guarantee there are no problems, of course... > > > Aliasing problems are seen much more frequently on PowerPC than any other > platform for Clang. I found this a while back when doing some Clang > testing, and I still see problems with upstream unless I explicitly set > -fno-strict-aliasing. Nathan had mentioned wanting to get upstream to use > -fno-strict-aliasing by default on all platforms, but I don't think that > ever made it beyond his suggesting. > > I filed this bug to track it: http://llvm.org/bugs/show_bug.cgi?id=11955 > > > In my experience, most C programmers misunderstand the aliasing rules of C > >> and even people on the C++ standards committee often get them wrong for > >> C++, so trading a 1-10% performance increase for a significant chance of > >> generating non-working code seems like a poor gain. If people are certain > >> that they do understand the rules, then they can add -fstrict-aliasing to > >> their own CFLAGS. > >> > > > > I'm actually quite interested in the performance difference; I think I > > will run a few tests. :) > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 17:44:02 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 74DE1106566B; Wed, 5 Sep 2012 17:44:02 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id D80F28FC15; Wed, 5 Sep 2012 17:44:01 +0000 (UTC) Received: by vcbgb30 with SMTP id gb30so1545039vcb.13 for ; Wed, 05 Sep 2012 10:44:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=kWoyoBwoF/EyLD0qwovGQLm8re1MFZ4fQ0tee8lTK+Y=; b=NKgVXFgM7Pay/x4bfcBTg0LZpWmQcpZWBhIcTbj8qBld//R5HNRAsbo3Q5JkIzJgcY VePxC9w4iranJtAidnrvBIEJwwWkH1p1j/MT1anBRerV13lLtSAkhgaMnneGFnBhTRd7 kG3miaYEHIbSRdbcFFtVvJ3GZFlgz8FbTznm64M5WnHIPhaWJ+ANHbtfbWxUlpw1qOIj DUvtFU52eYhnfh84XCB3/jGOGaHu1H/KQCOWyFwUjQOgl/KeAdnvsYIA8KIa/SN1RK/p ucM2+O+hgqd1crrAxDi5lDce7RRlNOdhxLw2oq5iQ+GoS5HVijRvRJxi09inbG2FE9O3 qioQ== MIME-Version: 1.0 Received: by 10.52.92.200 with SMTP id co8mr16553321vdb.131.1346867040779; Wed, 05 Sep 2012 10:44:00 -0700 (PDT) Received: by 10.58.203.136 with HTTP; Wed, 5 Sep 2012 10:44:00 -0700 (PDT) In-Reply-To: <20120905173704.GA31250@freebsd.org> References: <5046670C.6050500@andric.com> <20120904214344.GA17723@troutmask.apl.washington.edu> <504679CB.90204@andric.com> <20120904221413.GA19395@troutmask.apl.washington.edu> <50471BEE.6030708@andric.com> <96BD00DE-865C-4690-A2F1-E5B7C5D221C0@FreeBSD.org> <50472FC7.8010500@andric.com> <20120905173704.GA31250@freebsd.org> Date: Wed, 5 Sep 2012 13:44:00 -0400 Message-ID: From: Justin Hibbits To: Roman Divacky Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Garrett Cooper , Dimitry Andric , freebsd-current@freebsd.org, David Chisnall , Steve Kargl Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 17:44:02 -0000 Actually, Nathan does say it's gcc's fault in a comment on that bug. However, I do all my clang work compiling it with gcc4.2.1, so run into this constantly when I forget to add the flag. - Justin On Wed, Sep 5, 2012 at 1:37 PM, Roman Divacky wrote: > What makes you think it's a bug in llvm code and not a plain gcc > miscompile? > Other people seem to compile llvm on PPC64 with gcc and -fstrict-aliasing > just fine. They just dont happen to use gcc4.2.1. Ie. gcc47 is reported > to not have this problem. I personally can confirm that fbsd+gcc48 is ok to > > On Wed, Sep 05, 2012 at 09:11:22AM -0400, Justin Hibbits wrote: > > On Wed, Sep 5, 2012 at 6:56 AM, Dimitry Andric > wrote: > > > > > On 2012-09-05 11:36, David Chisnall wrote: > > > > > >> On 5 Sep 2012, at 10:31, Dimitry Andric wrote: > > >> > > >>> TThe > > >>> > > >>> -fno-strict-aliasing is not really my choice, but it was > introduced > > >>> in the past by Nathan Whitehorn, who apparently saw problems > without > > >>> it. It will hopefully disappear in the future. > > >>> > > >> Clang currently defaults to no strict aliasing on FreeBSD. > > >> > > > > > > Yes, but upstream has never used -fno-strict-aliasing, just plain -O2. > > > I run regular separate builds of pristine upstream clang on FreeBSD, > and > > > I haven't seen any failures due aliasing problems in all the regression > > > tests. That doesn't guarantee there are no problems, of course... > > > > > > Aliasing problems are seen much more frequently on PowerPC than any other > > platform for Clang. I found this a while back when doing some Clang > > testing, and I still see problems with upstream unless I explicitly set > > -fno-strict-aliasing. Nathan had mentioned wanting to get upstream to > use > > -fno-strict-aliasing by default on all platforms, but I don't think that > > ever made it beyond his suggesting. > > > > I filed this bug to track it: http://llvm.org/bugs/show_bug.cgi?id=11955 > > > > > > In my experience, most C programmers misunderstand the aliasing rules of > C > > >> and even people on the C++ standards committee often get them wrong > for > > >> C++, so trading a 1-10% performance increase for a significant > chance of > > >> generating non-working code seems like a poor gain. If people are > certain > > >> that they do understand the rules, then they can add > -fstrict-aliasing to > > >> their own CFLAGS. > > >> > > > > > > I'm actually quite interested in the performance difference; I think I > > > will run a few tests. :) > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to " > freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 17:45:12 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BADA41065670; Wed, 5 Sep 2012 17:45:12 +0000 (UTC) (envelope-from rdivacky@vlakno.cz) Received: from vlakno.cz (lev.vlakno.cz [46.28.110.116]) by mx1.freebsd.org (Postfix) with ESMTP id 5BD788FC16; Wed, 5 Sep 2012 17:45:12 +0000 (UTC) Received: by vlakno.cz (Postfix, from userid 1002) id 56FB77F3888; Wed, 5 Sep 2012 19:45:10 +0200 (CEST) Date: Wed, 5 Sep 2012 19:45:10 +0200 From: Roman Divacky To: Justin Hibbits Message-ID: <20120905174510.GA43305@freebsd.org> References: <20120904214344.GA17723@troutmask.apl.washington.edu> <504679CB.90204@andric.com> <20120904221413.GA19395@troutmask.apl.washington.edu> <50471BEE.6030708@andric.com> <96BD00DE-865C-4690-A2F1-E5B7C5D221C0@FreeBSD.org> <50472FC7.8010500@andric.com> <20120905173704.GA31250@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Garrett Cooper , Dimitry Andric , freebsd-current@freebsd.org, David Chisnall , Steve Kargl Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 17:45:12 -0000 I've been compiling clang with itself on PPC64 for a while now. Works quite good :) On Wed, Sep 05, 2012 at 01:44:00PM -0400, Justin Hibbits wrote: > Actually, Nathan does say it's gcc's fault in a comment on that bug. > However, I do all my clang work compiling it with gcc4.2.1, so run into > this constantly when I forget to add the flag. > > - Justin > > On Wed, Sep 5, 2012 at 1:37 PM, Roman Divacky wrote: > > > What makes you think it's a bug in llvm code and not a plain gcc > > miscompile? > > Other people seem to compile llvm on PPC64 with gcc and -fstrict-aliasing > > just fine. They just dont happen to use gcc4.2.1. Ie. gcc47 is reported > > to not have this problem. I personally can confirm that fbsd+gcc48 is ok to > > > > On Wed, Sep 05, 2012 at 09:11:22AM -0400, Justin Hibbits wrote: > > > On Wed, Sep 5, 2012 at 6:56 AM, Dimitry Andric > > wrote: > > > > > > > On 2012-09-05 11:36, David Chisnall wrote: > > > > > > > >> On 5 Sep 2012, at 10:31, Dimitry Andric wrote: > > > >> > > > >>> TThe > > > >>> > > > >>> -fno-strict-aliasing is not really my choice, but it was > > introduced > > > >>> in the past by Nathan Whitehorn, who apparently saw problems > > without > > > >>> it. It will hopefully disappear in the future. > > > >>> > > > >> Clang currently defaults to no strict aliasing on FreeBSD. > > > >> > > > > > > > > Yes, but upstream has never used -fno-strict-aliasing, just plain -O2. > > > > I run regular separate builds of pristine upstream clang on FreeBSD, > > and > > > > I haven't seen any failures due aliasing problems in all the regression > > > > tests. That doesn't guarantee there are no problems, of course... > > > > > > > > > Aliasing problems are seen much more frequently on PowerPC than any other > > > platform for Clang. I found this a while back when doing some Clang > > > testing, and I still see problems with upstream unless I explicitly set > > > -fno-strict-aliasing. Nathan had mentioned wanting to get upstream to > > use > > > -fno-strict-aliasing by default on all platforms, but I don't think that > > > ever made it beyond his suggesting. > > > > > > I filed this bug to track it: http://llvm.org/bugs/show_bug.cgi?id=11955 > > > > > > > > > In my experience, most C programmers misunderstand the aliasing rules of > > C > > > >> and even people on the C++ standards committee often get them wrong > > for > > > >> C++, so trading a 1-10% performance increase for a significant > > chance of > > > >> generating non-working code seems like a poor gain. If people are > > certain > > > >> that they do understand the rules, then they can add > > -fstrict-aliasing to > > > >> their own CFLAGS. > > > >> > > > > > > > > I'm actually quite interested in the performance difference; I think I > > > > will run a few tests. :) > > > _______________________________________________ > > > freebsd-current@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > > To unsubscribe, send any mail to " > > freebsd-current-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 17:59:57 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 87DBD106566B for ; Wed, 5 Sep 2012 17:59:57 +0000 (UTC) (envelope-from eirnym@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 525B78FC16 for ; Wed, 5 Sep 2012 17:59:57 +0000 (UTC) Received: by iayy25 with SMTP id y25so1404210iay.13 for ; Wed, 05 Sep 2012 10:59:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=LOoeQJ0GW3hZBZNxPs8dcQf6CVHev6cfaNBzCNq9iVg=; b=DOea4xWQMEC4/A4nDbxd61LVDyIxE7foIL/y6sOTkpwod9R5OURbuCSh69SqWprefX VMkhGMrjK20Z8njBPxCnriB2Tb0mK6PVYEWW+VE4Jxc96vbBeX+Qw0WVmpshrUcotimD Ux3krbin1a/2CzqdUFRgBybJvsj+QsEKyjknDXRY1Rgas+xmVgx0b5TPwRbyEgOVdgoK WC9y+OIa6/z+eWEyg8aboNYcI4qk5ipvDXHcOrTJ2Ix96tkkVdSl4WOBE6JjAAhuuv/q mbrqQdHG/obyzhazgN0+GkaVhPd9gNk5pnLgseLlux9FtTzB/hqlQAhNJ197koaigEUy dotw== Received: by 10.50.168.105 with SMTP id zv9mr19120800igb.73.1346867996578; Wed, 05 Sep 2012 10:59:56 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.46.41 with HTTP; Wed, 5 Sep 2012 10:59:36 -0700 (PDT) From: Eir Nym Date: Wed, 5 Sep 2012 21:59:36 +0400 Message-ID: To: FreeBSD Mail Lists Content-Type: text/plain; charset=UTF-8 Subject: Are clang unsigned comparison warnings in kern/kern_* ok? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 17:59:57 -0000 I've got following warnings [no errors had been generated while -Werror is in command line] and want to know if they are ok. There are much more same warnings in modules, but I worry about kernel : Kernel config: http://eroese.org/_/_/pub/bsd/240070/GENERIC_PF.amd64 src.conf: http://eroese.org/_/_/pub/bsd/240070/src.conf make.conf: /dev/null kernel build logs: http://eroese.org/_/_/pub/bsd/240070/kernel.amd64.GENERIC_PF cc -c -O2 -pipe -fno-strict-aliasing -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-parentheses-equality -nostdinc -I. -I/usr/head/src/sys -I/usr/head/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-omit-frame-pointer -mno-aes -mno-avx -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector -Werror /usr/head/src/sys/kern/kern_cpuset.c /usr/head/src/sys/kern/kern_cpuset.c:654:16: warning: comparison of unsigned expression < 0 is always false [-Wtautological-compare] for (i = 0; i < (_NCPUWORDS - 1); i++) { ~ ^ ~~~~~~~~~~~~~~~~ 1 warning generated. /usr/head/src/sys/kern/kern_poll.c:173:10: warning: comparison of unsigned expression < 0 is always false [-Wtautological-compare] if (val < 0 || val > 99) ~~~ ^ ~ 1 warning generated. cc -c -O2 -pipe -fno-strict-aliasing -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-parentheses-equality -nostdinc -I. -I/usr/head/src/sys -I/usr/head/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-omit-frame-pointer -mno-aes -mno-avx -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector -Werror /usr/head/src/sys/kern/kern_umtx.c /usr/head/src/sys/kern/kern_umtx.c:3312:19: warning: comparison of unsigned expression < 0 is always false [-Wtautological-compare] if (ts32.tv_sec < 0 || ~~~~~~~~~~~ ^ ~ /usr/head/src/sys/kern/kern_umtx.c:3314:20: warning: comparison of unsigned expression < 0 is always false [-Wtautological-compare] ts32.tv_nsec < 0) ~~~~~~~~~~~~ ^ ~ /usr/head/src/sys/kern/kern_umtx.c:3338:25: warning: comparison of unsigned expression < 0 is always false [-Wtautological-compare] if (t32.timeout.tv_sec < 0 || ~~~~~~~~~~~~~~~~~~ ^ ~ /usr/head/src/sys/kern/kern_umtx.c:3339:63: warning: comparison of unsigned expression < 0 is always false [-Wtautological-compare] t32.timeout.tv_nsec >= 1000000000 || t32.timeout.tv_nsec < 0) ~~~~~~~~~~~~~~~~~~~ ^ ~ 4 warnings generated. cc -c -O2 -pipe -fno-strict-aliasing -std=c99 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-parentheses-equality -nostdinc -I. -I/usr/head/src/sys -I/usr/head/src/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-omit-frame-pointer -mno-aes -mno-avx -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector -Werror /usr/head/src/sys/kern/uipc_syscalls.c /usr/head/src/sys/kern/uipc_syscalls.c:356:16: warning: comparison of unsigned expression < 0 is always false [-Wtautological-compare] if (*namelen < 0) ~~~~~~~~ ^ ~ /usr/head/src/sys/kern/uipc_syscalls.c:1487:12: warning: comparison of unsigned expression < 0 is always false [-Wtautological-compare] if (*alen < 0) ~~~~~ ^ ~ /usr/head/src/sys/kern/uipc_syscalls.c:1587:12: warning: comparison of unsigned expression < 0 is always false [-Wtautological-compare] if (*alen < 0) ~~~~~ ^ ~ 3 warnings generated. -- Eir Nym From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 18:12:39 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B7A1A106566B for ; Wed, 5 Sep 2012 18:12:39 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 2545D8FC14 for ; Wed, 5 Sep 2012 18:12:38 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 0C030138EA; Thu, 6 Sep 2012 04:12:32 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro.local (c-75-70-8-22.hsd1.co.comcast.net [75.70.8.22]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BGG36879 (AUTH peterg@ptree32.com.au); Thu, 6 Sep 2012 04:12:29 +1000 Message-ID: <5047960A.5060102@freebsd.org> Date: Wed, 05 Sep 2012 12:12:26 -0600 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.28) Gecko/20120306 Thunderbird/3.1.20 MIME-Version: 1.0 To: fabien.thomas@netasq.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Junkmail-Info: RCVD_IN_PBL, RCVD_IN_SORBS_DUL, RDNS_DYNAMIC, SPF_SOFTFAIL, UPPERCASE_75_100 X-Junkmail-Status: score=48/51, host=dommail.onthenet.com.au Cc: current@freebsd.org Subject: re: [CFT] hwpmc support for Intel Ivy Bridge X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 18:12:39 -0000 Another system: CPU: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (3392.36-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x306a9 Family = 6 Model = 3a Stepping = 9 Features=0xbfebfbff Features2=0x7fbae3ff AMD Features=0x28100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics All pmcstat tests worked. The pmctest.py script failed with SOFT the same way as bapt@'s, but all subsequent tests passed. pmccontrol -L output appended. later, Peter. # ./pmccontrol -L SOFT CLOCK.STAT CLOCK.HARD LOCK.FAILED PAGE_FAULT.WRITE PAGE_FAULT.READ PAGE_FAULT.ALL TSC TSC IAP LD_BLOCKS.STORE_FORWARD MISALIGN_MEM_REF.LOADS MISALIGN_MEM_REF.STORES LD_BLOCKS_PARTIAL.ADDRESS_ALIAS DTLB_LOAD_MISSES.DEMAND_LD_MISS_CAUSES_A_WALK DTLB_LOAD_MISSES.DEMAND_LD_WALK_COMPLETED DTLB_LOAD_MISSES.DEMAND_LD_WALK_DURATION UOPS_ISSUED.ANY UOPS_ISSUED.FLAGS_MERGE UOPS_ISSUED.SLOW_LEA UOPS_ISSUED.SINGLE_MUL ARITH.FPU_DIV_ACTIVE L2_RQSTS.DEMAND_DATA_RD_HIT L2_RQSTS.ALL_DEMAND_DATA_RD L2_RQSTS.RFO_HITS L2_RQSTS.RFO_MISS L2_RQSTS.ALL_RFO L2_RQSTS.CODE_RD_HIT L2_RQSTS.CODE_RD_MISS L2_RQSTS.ALL_CODE_RD L2_RQSTS.PF_HIT L2_RQSTS.PF_MISS L2_RQSTS.ALL_PF L2_STORE_LOCK_RQSTS.MISS L2_STORE_LOCK_RQSTS.HIT_M L2_STORE_LOCK_RQSTS.ALL L2_L1D_WB_RQSTS.MISS L2_L1D_WB_RQSTS.HIT_E L2_L1D_WB_RQSTS.HIT_M L2_L1D_WB_RQSTS.ALL LONGEST_LAT_CACHE.REFERENCE LONGEST_LAT_CACHE.MISS CPU_CLK_UNHALTED.THREAD_P CPU_CLK_THREAD_UNHALTED.REF_XCLK L1D_PEND_MISS.PENDING DTLB_STORE_MISSES.MISS_CAUSES_A_WALK DTLB_STORE_MISSES.WALK_COMPLETED DTLB_STORE_MISSES.WALK_DURATION DTLB_STORE_MISSES.STLB_HIT LOAD_HIT_PRE.SW_PF LOAD_HIT_PRE.HW_PF L1D.REPLACEMENT MOVE_ELIMINATION.INT_NOT_ELIMINATED MOVE_ELIMINATION.SIMD_NOT_ELIMINATED MOVE_ELIMINATION.INT_ELIMINATED MOVE_ELIMINATION.SIMD_ELIMINATED CPL_CYCLES.RING0 CPL_CYCLES.RING123 RS_EVENTS.EMPTY_CYCLES TLB_ACCESS.LOAD_STLB_HIT OFFCORE_REQUESTS_OUTSTANDING.DEMAND_DATA_RD OFFCORE_REQUESTS_OUTSTANDING.DEMAND_CODE_RD OFFCORE_REQUESTS_OUTSTANDING.DEMAND_RFO OFFCORE_REQUESTS_OUTSTANDING.ALL_DATA_RD LOCK_CYCLES.SPLIT_LOCK_UC_LOCK_DURATION LOCK_CYCLES.CACHE_LOCK_DURATION IDQ.EMPTY IDQ.MITE_UOPS IDQ.DSB_UOPS IDQ.MS_DSB_UOPS IDQ.MS_MITE_UOPS IDQ.MS_UOPS IDQ.ALL_DSB_CYCLES_ANY_UOPS IDQ.ALL_DSB_CYCLES_4_UOPS IDQ.ALL_MITE_CYCLES_ANY_UOPS IDQ.ALL_MITE_CYCLES_4_UOPS IDQ.MITE_ALL_UOPS ICACHE.MISSES ITLB_MISSES.MISS_CAUSES_A_WALK ITLB_MISSES.WALK_COMPLETED ITLB_MISSES.WALK_DURATION ITLB_MISSES.STLB_HIT ILD_STALL.LCP ILD_STALL.IQ_FULL BR_INST_EXEC.COND BR_INST_EXEC.DIRECT_JMP BR_INST_EXEC.INDIRECT_JMP_NON_CALL_RET BR_INST_EXEC.RETURN_NEAR BR_INST_EXEC.DIRECT_NEAR_CALL BR_INST_EXEC.INDIRECT_NEAR_CALL BR_INST_EXEC.NONTAKEN BR_INST_EXEC.TAKEN BR_INST_EXEC.ALL_BRANCHES BR_MISP_EXEC.COND BR_MISP_EXEC.INDIRECT_JMP_NON_CALL_RET BR_MISP_EXEC.RETURN_NEAR BR_MISP_EXEC.DIRECT_NEAR_CALL BR_MISP_EXEC.INDIRECT_NEAR_CALL BR_MISP_EXEC.NONTAKEN BR_MISP_EXEC.TAKEN BR_MISP_EXEC.ALL_BRANCHES IDQ_UOPS_NOT_DELIVERED.CORE UOPS_DISPATCHED_PORT.PORT_0 UOPS_DISPATCHED_PORT.PORT_1 UOPS_DISPATCHED_PORT.PORT_2_LD UOPS_DISPATCHED_PORT.PORT_2_STA UOPS_DISPATCHED_PORT.PORT_2 UOPS_DISPATCHED_PORT.PORT_3_LD UOPS_DISPATCHED_PORT.PORT_3_STA UOPS_DISPATCHED_PORT.PORT_3 UOPS_DISPATCHED_PORT.PORT_4 UOPS_DISPATCHED_PORT.PORT_5 RESOURCE_STALLS.ANY RESOURCE_STALLS.RS RESOURCE_STALLS.SB RESOURCE_STALLS.ROB DSB2MITE_SWITCHES.COUNT DSB2MITE_SWITCHES.PENALTY_CYCLES DSB_FILL.EXCEED_DSB_LINES ITLB.ITLB_FLUSH OFFCORE_REQUESTS.DEMAND_DATA_RD OFFCORE_REQUESTS.DEMAND_CODE_RD OFFCORE_REQUESTS.DEMAND_RFO OFFCORE_REQUESTS.ALL_DATA_RD UOPS_EXECUTED.THREAD UOPS_EXECUTED.CORE OFF_CORE_RESPONSE_0 OFF_CORE_RESPONSE_1 TLB_FLUSH.DTLB_THREAD TLB_FLUSH.STLB_ANY INST_RETIRED.ANY_P INST_RETIRED.ALL OTHER_ASSISTS.AVX_STORE OTHER_ASSISTS.AVX_TO_SSE OTHER_ASSISTS.SSE_TO_AVX UOPS_RETIRED.ALL UOPS_RETIRED.RETIRE_SLOTS MACHINE_CLEARS.MEMORY_ORDERING MACHINE_CLEARS.SMC MACHINE_CLEARS.MASKMOV BR_INST_RETIRED.ALL_BRANCHES BR_INST_RETIRED.CONDITIONAL BR_INST_RETIRED.NEAR_CALL BR_INST_RETIRED.ALL_BRANCHES BR_INST_RETIRED.NEAR_RETURN BR_INST_RETIRED.NOT_TAKEN BR_INST_RETIRED.NEAR_TAKEN BR_INST_RETIRED.FAR_BRANCH BR_MISP_RETIRED.ALL_BRANCHES BR_MISP_RETIRED.CONDITIONAL BR_MISP_RETIRED.NEAR_CALL BR_MISP_RETIRED.ALL_BRANCHES BR_MISP_RETIRED.NOT_TAKEN BR_MISP_RETIRED.TAKEN FP_ASSIST.X87_OUTPUT FP_ASSIST.X87_INPUT FP_ASSIST.SIMD_OUTPUT FP_ASSIST.SIMD_INPUT FP_ASSIST.ANY ROB_MISC_EVENTS.LBR_INSERTS MEM_TRANS_RETIRED.LOAD_LATENCY MEM_TRANS_RETIRED.PRECISE_STORE MEM_UOP_RETIRED.LOADS MEM_UOP_RETIRED.STORES MEM_UOP_RETIRED.STLB_MISS MEM_UOP_RETIRED.LOCK MEM_UOP_RETIRED.SPLIT MEM_UOP_RETIRED.ALL MEM_LOAD_UOPS_RETIRED.L1_HIT MEM_LOAD_UOPS_RETIRED.L2_HIT MEM_LOAD_UOPS_RETIRED.LLC_HIT MEM_LOAD_UOPS_RETIRED.HIT_LFB MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_MISS MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_NONE MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM L2_TRANS.DEMAND_DATA_RD L2_TRANS.RFO L2_TRANS.CODE_RD L2_TRANS.ALL_PF L2_TRANS.L1D_WB L2_TRANS.L2_FILL L2_TRANS.L2_WB L2_TRANS.ALL_REQUESTS L2_LINES_IN.I L2_LINES_IN.S L2_LINES_IN.E L2_LINES_IN.ALL L2_LINES_OUT.DEMAND_CLEAN L2_LINES_OUT.DEMAND_DIRTY L2_LINES_OUT.PF_CLEAN L2_LINES_OUT.PF_DIRTY IAF INSTR_RETIRED_ANY CPU_CLK_UNHALTED_CORE CPU_CLK_UNHALTED_REF From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 20:25:30 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9D2F7106566C for ; Wed, 5 Sep 2012 20:25:30 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id 58B3B8FC0C for ; Wed, 5 Sep 2012 20:25:30 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:d43c:3039:49b5:2229] (unknown [IPv6:2001:7b8:3a7:0:d43c:3039:49b5:2229]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 8EE855C43; Wed, 5 Sep 2012 22:25:29 +0200 (CEST) Message-ID: <5047B539.3030000@FreeBSD.org> Date: Wed, 05 Sep 2012 22:25:29 +0200 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20120828 Thunderbird/16.0 MIME-Version: 1.0 To: Eir Nym References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: Are clang unsigned comparison warnings in kern/kern_* ok? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 20:25:30 -0000 On 2012-09-05 19:59, Eir Nym wrote: > I've got following warnings [no errors had been generated while > -Werror is in command line] and want to know if they are ok. Most of these warnings are harmless, and just point out that the compiler will optimize unused code away, such as tests that always succeed or fail. In some cases it would be nice to fix them, but there is usually no urgent need. > /usr/head/src/sys/kern/kern_cpuset.c:654:16: warning: comparison of > unsigned expression < 0 is always false [-Wtautological-compare] > for (i = 0; i < (_NCPUWORDS - 1); i++) { > ~ ^ ~~~~~~~~~~~~~~~~ For example, this warning is harmless. It turns out _NCPUWORDS is 1, at least by default, on i386 and amd64. So the warning is correct, in the sense that the body of the for loop will never be executed. It will almost certainly be optimized away. Maybe the loop could be enclosed in a #if _NCPUWORDS > 1 clause, to prevent this warning. Also note, this warning only appeared after r239923, when the for loop was reversed. > /usr/head/src/sys/kern/kern_poll.c:173:10: warning: comparison of > unsigned expression < 0 is always false [-Wtautological-compare] > if (val < 0 || val > 99) > ~~~ ^ ~ This one might be fixed, since 'val' is declared as uint32_t. > /usr/head/src/sys/kern/kern_umtx.c:3312:19: warning: comparison of > unsigned expression < 0 is always false [-Wtautological-compare] > if (ts32.tv_sec < 0 || > ~~~~~~~~~~~ ^ ~ It looks like this part of kern_umtx.c (used for COMPAT_FREEBSD32) declares its own version of struct timespec32, where the tv_sec member is uint32_t. Therefore, this particular test is superfluous. From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 20:34:50 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 86AE01065676; Wed, 5 Sep 2012 20:34:50 +0000 (UTC) Date: Wed, 5 Sep 2012 20:34:50 +0000 From: Alexander Best To: Eir Nym Message-ID: <20120905203450.GA45200@freebsd.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Cc: FreeBSD Mail Lists Subject: Re: Are clang unsigned comparison warnings in kern/kern_* ok? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 20:34:50 -0000 On Wed Sep 5 12, Eir Nym wrote: > I've got following warnings [no errors had been generated while > -Werror is in command line] and want to know if they are ok. There are > much more same warnings in modules, but I worry about kernel : > Kernel config: http://eroese.org/_/_/pub/bsd/240070/GENERIC_PF.amd64 > src.conf: http://eroese.org/_/_/pub/bsd/240070/src.conf > make.conf: /dev/null > kernel build logs: http://eroese.org/_/_/pub/bsd/240070/kernel.amd64.GENERIC_PF yes these are OK. clang complains about variables that were defined as unsigned, but are being compared against < 0. since those variables cannot be negative, comparing them against < 0 makes no sence (hence -Wtautological-compare as name to define this group of warnings). in theory calng is right, however in practice there might be good reasons to do so: 1) as a general safety belt to make programmers feel comfortable 2) the check might be there, because in the future it is being planed to switch a certain variable from unsigned to signed 3) a variable type might be unsigned on one platform per default (no signed or unsigned keyword used), but might be signed on another. doing the < 0 check prevents programmers from having to deal with both cases. you can disable these warnings for clang by setting -Wno-tautological-compare. no idea however what var to put it in. CWARNFLAGS+= in src.conf maybe. cheers. alex > > cc -c -O2 -pipe -fno-strict-aliasing -std=c99 -Wall > -Wredundant-decls -Wnested-externs -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef > -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs > -fdiagnostics-show-option -Wno-error-tautological-compare > -Wno-error-empty-body -Wno-error-parentheses-equality -nostdinc -I. > -I/usr/head/src/sys -I/usr/head/src/sys/contrib/altq -D_KERNEL > -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h > -fno-omit-frame-pointer -mno-aes -mno-avx -mcmodel=kernel > -mno-red-zone -mno-mmx -mno-sse -msoft-float > -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector > -Werror /usr/head/src/sys/kern/kern_cpuset.c > /usr/head/src/sys/kern/kern_cpuset.c:654:16: warning: comparison of > unsigned expression < 0 is always false [-Wtautological-compare] > for (i = 0; i < (_NCPUWORDS - 1); i++) { > ~ ^ ~~~~~~~~~~~~~~~~ > 1 warning generated. > > /usr/head/src/sys/kern/kern_poll.c:173:10: warning: comparison of > unsigned expression < 0 is always false [-Wtautological-compare] > if (val < 0 || val > 99) > ~~~ ^ ~ > 1 warning generated. > > cc -c -O2 -pipe -fno-strict-aliasing -std=c99 -Wall > -Wredundant-decls -Wnested-externs -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef > -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs > -fdiagnostics-show-option -Wno-error-tautological-compare > -Wno-error-empty-body -Wno-error-parentheses-equality -nostdinc -I. > -I/usr/head/src/sys -I/usr/head/src/sys/contrib/altq -D_KERNEL > -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h > -fno-omit-frame-pointer -mno-aes -mno-avx -mcmodel=kernel > -mno-red-zone -mno-mmx -mno-sse -msoft-float > -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector > -Werror /usr/head/src/sys/kern/kern_umtx.c > /usr/head/src/sys/kern/kern_umtx.c:3312:19: warning: comparison of > unsigned expression < 0 is always false [-Wtautological-compare] > if (ts32.tv_sec < 0 || > ~~~~~~~~~~~ ^ ~ > /usr/head/src/sys/kern/kern_umtx.c:3314:20: warning: comparison of > unsigned expression < 0 is always false [-Wtautological-compare] > ts32.tv_nsec < 0) > ~~~~~~~~~~~~ ^ ~ > /usr/head/src/sys/kern/kern_umtx.c:3338:25: warning: comparison of > unsigned expression < 0 is always false [-Wtautological-compare] > if (t32.timeout.tv_sec < 0 || > ~~~~~~~~~~~~~~~~~~ ^ ~ > /usr/head/src/sys/kern/kern_umtx.c:3339:63: warning: comparison of > unsigned expression < 0 is always false [-Wtautological-compare] > t32.timeout.tv_nsec >= 1000000000 || t32.timeout.tv_nsec < 0) > ~~~~~~~~~~~~~~~~~~~ ^ ~ > 4 warnings generated. > > > cc -c -O2 -pipe -fno-strict-aliasing -std=c99 -Wall > -Wredundant-decls -Wnested-externs -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef > -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs > -fdiagnostics-show-option -Wno-error-tautological-compare > -Wno-error-empty-body -Wno-error-parentheses-equality -nostdinc -I. > -I/usr/head/src/sys -I/usr/head/src/sys/contrib/altq -D_KERNEL > -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h > -fno-omit-frame-pointer -mno-aes -mno-avx -mcmodel=kernel > -mno-red-zone -mno-mmx -mno-sse -msoft-float > -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector > -Werror /usr/head/src/sys/kern/uipc_syscalls.c > /usr/head/src/sys/kern/uipc_syscalls.c:356:16: warning: comparison of > unsigned expression < 0 is always false [-Wtautological-compare] > if (*namelen < 0) > ~~~~~~~~ ^ ~ > /usr/head/src/sys/kern/uipc_syscalls.c:1487:12: warning: comparison of > unsigned expression < 0 is always false [-Wtautological-compare] > if (*alen < 0) > ~~~~~ ^ ~ > /usr/head/src/sys/kern/uipc_syscalls.c:1587:12: warning: comparison of > unsigned expression < 0 is always false [-Wtautological-compare] > if (*alen < 0) > ~~~~~ ^ ~ > 3 warnings generated. > > -- Eir Nym From owner-freebsd-current@FreeBSD.ORG Wed Sep 5 22:13:23 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9FA2F106566C for ; Wed, 5 Sep 2012 22:13:23 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) by mx1.freebsd.org (Postfix) with ESMTP id 7845F8FC0C for ; Wed, 5 Sep 2012 22:13:23 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.5/8.14.5) with ESMTP id q85MDBBS097934; Wed, 5 Sep 2012 15:13:11 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.5/8.14.5/Submit) id q85MDBRI097933; Wed, 5 Sep 2012 15:13:11 -0700 (PDT) (envelope-from sgk) Date: Wed, 5 Sep 2012 15:13:11 -0700 From: Steve Kargl To: Dimitry Andric Message-ID: <20120905221310.GA97847@troutmask.apl.washington.edu> References: <5046670C.6050500@andric.com> <20120904214344.GA17723@troutmask.apl.washington.edu> <504679CB.90204@andric.com> <20120904221413.GA19395@troutmask.apl.washington.edu> <50471BEE.6030708@andric.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50471BEE.6030708@andric.com> User-Agent: Mutt/1.4.2.3i Cc: Garrett Cooper , freebsd-current@freebsd.org Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Sep 2012 22:13:23 -0000 On Wed, Sep 05, 2012 at 11:31:26AM +0200, Dimitry Andric wrote: > On 2012-09-05 01:40, Garrett Cooper wrote: > ... > > Steve does have a point. Posting the results of > >CFLAGS/CPPFLAGS/LDFLAGS/etc for config.log (and maybe poking through > >the code to figure out what *FLAGS were used elsewhere) is more > >valuable than the data is in its current state (unfortunately.. > >autoconf makes things more complicated). > > 1) For building the FreeBSD in-tree version of clang 3.2: > > -O2 -pipe -fno-strict-aliasing > > 2) For building the FreeBSD in-tree version of gcc 4.2.1: > > -O2 -pipe > > 3) For building Boost 1.50.0: > > -ftemplate-depth-128 -O3 -finline-functions > Dimitry thanks for the follow-up. I performed an unscientific (micro)benchmark of /usr/bin/cc vs /usr/bin/clang where cc is the base system's gcc 4.2.1. Here's what I found/feared. Compiling libm on CPU: AMD Opteron(tm) Processor 248 (2192.01-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0xf5a Family = f Model = 5 Stepping = 10 Features=0x78bfbff AMD Features=0xe0500800 with default CFLAGS (ie., -O2 -pipe) and -march=opteron. Using 'setenv CC /usr/bin/cc' with 3 runs of make clean time make -DNO_MAN yields 69.39 real 52.00 user 38.55 sys 69.57 real 52.35 user 38.37 sys 69.48 real 52.25 user 38.38 sys Now, repeating with 'setenv CC /usr/bin/clang' yields 39.65 real 21.86 user 17.37 sys 40.91 real 21.48 user 17.91 sys 39.77 real 21.65 user 17.64 sys So, clang does appear to be faster in this particular compiling speed benchmark. However, if I know build my test program for libm's j0f() function where the only difference is whether libm was built with /usr/bin/cc or /usr/bin/clang, I observe the following results. 1234567 x values in the interval [0:25] gcc libm | clang libm ----------------|----------------- ULP <= 0.6 --> 565515 (45.81%) | 513763 (41.61%) 0.6 < ULP <= 0.7 --> 74148 ( 6.01%) | 67221 ( 5.44%) 0.7 < ULP <= 0.8 --> 69112 ( 5.60%) | 62846 ( 5.09%) 0.8 < ULP <= 0.9 --> 63798 ( 5.17%) | 58217 ( 4.72%) 0.9 < ULP <= 1.0 --> 58679 ( 4.75%) | 53834 ( 4.36%) 1.0 < ULP <= 2.0 --> 328221 (26.59%) | 306728 (24.84%) 2.0 < ULP <= 3.0 --> 65323 ( 5.29%) | 63452 ( 5.14%) 3.0 < ULP --> 9771 ( 0.79%) | 108506 ( 8.79%) gcc libm | clang libm -----------------------|-------------------- MAX ULP: 12152.27637 | 1129606938624.00000 x at MAX ULP: 5.520077 0x1.6148f2p+2 | 2.404833 0x1.33d19p+1 Speed test with gcc libm. 1234567 j0f calls in 0.193427 seconds. 1234567 j0f calls in 0.193410 seconds. 1234567 j0f calls in 0.194158 seconds. Speed test with clang libm. 1234567 j0f calls in 0.180260 seconds. 1234567 j0f calls in 0.180130 seconds. 1234567 j0f calls in 0.179739 seconds. So, although the clang built j0f() appears to be faster than the gcc built j0f(), the clang built j0f() has much worse accuracy issues. -- Steve From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 06:20:02 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BB647106564A; Thu, 6 Sep 2012 06:20:02 +0000 (UTC) (envelope-from fabien.thomas@netasq.com) Received: from work.netasq.com (gwlille.netasq.com [91.212.116.1]) by mx1.freebsd.org (Postfix) with ESMTP id 433EF8FC08; Thu, 6 Sep 2012 06:20:02 +0000 (UTC) Received: from [10.2.1.1] (unknown [10.2.1.1]) by work.netasq.com (Postfix) with ESMTPSA id 4B42F27044BC; Thu, 6 Sep 2012 08:20:01 +0200 (CEST) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=iso-8859-1 From: Fabien Thomas In-Reply-To: <5047960A.5060102@freebsd.org> Date: Thu, 6 Sep 2012 08:20:00 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <6C3C65CF-C621-4DEE-84DF-4655EC788552@netasq.com> References: <5047960A.5060102@freebsd.org> To: Peter Grehan X-Mailer: Apple Mail (2.1278) Cc: current@freebsd.org Subject: Re: [CFT] hwpmc support for Intel Ivy Bridge X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 06:20:02 -0000 Le 5 sept. 2012 =E0 20:12, Peter Grehan a =E9crit : > Another system: >=20 > CPU: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (3392.36-MHz K8-class = CPU) > Origin =3D "GenuineIntel" Id =3D 0x306a9 Family =3D 6 Model =3D 3a = Stepping =3D 9 > = Features=3D0xbfebfbff > = Features2=3D0x7fbae3ff > AMD Features=3D0x28100800 > AMD Features2=3D0x1 > TSC: P-state invariant, performance statistics >=20 > All pmcstat tests worked. The pmctest.py script failed with SOFT the = same way as bapt@'s, but all subsequent tests passed. Thanks!=20 Not a problem, the script parse pmccontrol and run pmcstat with each = value except section but SOFT is not in the exclusion list. >=20 > pmccontrol -L output appended. >=20 > later, >=20 > Peter. >=20 > # ./pmccontrol -L > SOFT > CLOCK.STAT > CLOCK.HARD > LOCK.FAILED > PAGE_FAULT.WRITE > PAGE_FAULT.READ > PAGE_FAULT.ALL > TSC > TSC > IAP > LD_BLOCKS.STORE_FORWARD > MISALIGN_MEM_REF.LOADS > MISALIGN_MEM_REF.STORES > LD_BLOCKS_PARTIAL.ADDRESS_ALIAS > DTLB_LOAD_MISSES.DEMAND_LD_MISS_CAUSES_A_WALK > DTLB_LOAD_MISSES.DEMAND_LD_WALK_COMPLETED > DTLB_LOAD_MISSES.DEMAND_LD_WALK_DURATION > UOPS_ISSUED.ANY > UOPS_ISSUED.FLAGS_MERGE > UOPS_ISSUED.SLOW_LEA > UOPS_ISSUED.SINGLE_MUL > ARITH.FPU_DIV_ACTIVE > L2_RQSTS.DEMAND_DATA_RD_HIT > L2_RQSTS.ALL_DEMAND_DATA_RD > L2_RQSTS.RFO_HITS > L2_RQSTS.RFO_MISS > L2_RQSTS.ALL_RFO > L2_RQSTS.CODE_RD_HIT > L2_RQSTS.CODE_RD_MISS > L2_RQSTS.ALL_CODE_RD > L2_RQSTS.PF_HIT > L2_RQSTS.PF_MISS > L2_RQSTS.ALL_PF > L2_STORE_LOCK_RQSTS.MISS > L2_STORE_LOCK_RQSTS.HIT_M > L2_STORE_LOCK_RQSTS.ALL > L2_L1D_WB_RQSTS.MISS > L2_L1D_WB_RQSTS.HIT_E > L2_L1D_WB_RQSTS.HIT_M > L2_L1D_WB_RQSTS.ALL > LONGEST_LAT_CACHE.REFERENCE > LONGEST_LAT_CACHE.MISS > CPU_CLK_UNHALTED.THREAD_P > CPU_CLK_THREAD_UNHALTED.REF_XCLK > L1D_PEND_MISS.PENDING > DTLB_STORE_MISSES.MISS_CAUSES_A_WALK > DTLB_STORE_MISSES.WALK_COMPLETED > DTLB_STORE_MISSES.WALK_DURATION > DTLB_STORE_MISSES.STLB_HIT > LOAD_HIT_PRE.SW_PF > LOAD_HIT_PRE.HW_PF > L1D.REPLACEMENT > MOVE_ELIMINATION.INT_NOT_ELIMINATED > MOVE_ELIMINATION.SIMD_NOT_ELIMINATED > MOVE_ELIMINATION.INT_ELIMINATED > MOVE_ELIMINATION.SIMD_ELIMINATED > CPL_CYCLES.RING0 > CPL_CYCLES.RING123 > RS_EVENTS.EMPTY_CYCLES > TLB_ACCESS.LOAD_STLB_HIT > OFFCORE_REQUESTS_OUTSTANDING.DEMAND_DATA_RD > OFFCORE_REQUESTS_OUTSTANDING.DEMAND_CODE_RD > OFFCORE_REQUESTS_OUTSTANDING.DEMAND_RFO > OFFCORE_REQUESTS_OUTSTANDING.ALL_DATA_RD > LOCK_CYCLES.SPLIT_LOCK_UC_LOCK_DURATION > LOCK_CYCLES.CACHE_LOCK_DURATION > IDQ.EMPTY > IDQ.MITE_UOPS > IDQ.DSB_UOPS > IDQ.MS_DSB_UOPS > IDQ.MS_MITE_UOPS > IDQ.MS_UOPS > IDQ.ALL_DSB_CYCLES_ANY_UOPS > IDQ.ALL_DSB_CYCLES_4_UOPS > IDQ.ALL_MITE_CYCLES_ANY_UOPS > IDQ.ALL_MITE_CYCLES_4_UOPS > IDQ.MITE_ALL_UOPS > ICACHE.MISSES > ITLB_MISSES.MISS_CAUSES_A_WALK > ITLB_MISSES.WALK_COMPLETED > ITLB_MISSES.WALK_DURATION > ITLB_MISSES.STLB_HIT > ILD_STALL.LCP > ILD_STALL.IQ_FULL > BR_INST_EXEC.COND > BR_INST_EXEC.DIRECT_JMP > BR_INST_EXEC.INDIRECT_JMP_NON_CALL_RET > BR_INST_EXEC.RETURN_NEAR > BR_INST_EXEC.DIRECT_NEAR_CALL > BR_INST_EXEC.INDIRECT_NEAR_CALL > BR_INST_EXEC.NONTAKEN > BR_INST_EXEC.TAKEN > BR_INST_EXEC.ALL_BRANCHES > BR_MISP_EXEC.COND > BR_MISP_EXEC.INDIRECT_JMP_NON_CALL_RET > BR_MISP_EXEC.RETURN_NEAR > BR_MISP_EXEC.DIRECT_NEAR_CALL > BR_MISP_EXEC.INDIRECT_NEAR_CALL > BR_MISP_EXEC.NONTAKEN > BR_MISP_EXEC.TAKEN > BR_MISP_EXEC.ALL_BRANCHES > IDQ_UOPS_NOT_DELIVERED.CORE > UOPS_DISPATCHED_PORT.PORT_0 > UOPS_DISPATCHED_PORT.PORT_1 > UOPS_DISPATCHED_PORT.PORT_2_LD > UOPS_DISPATCHED_PORT.PORT_2_STA > UOPS_DISPATCHED_PORT.PORT_2 > UOPS_DISPATCHED_PORT.PORT_3_LD > UOPS_DISPATCHED_PORT.PORT_3_STA > UOPS_DISPATCHED_PORT.PORT_3 > UOPS_DISPATCHED_PORT.PORT_4 > UOPS_DISPATCHED_PORT.PORT_5 > RESOURCE_STALLS.ANY > RESOURCE_STALLS.RS > RESOURCE_STALLS.SB > RESOURCE_STALLS.ROB > DSB2MITE_SWITCHES.COUNT > DSB2MITE_SWITCHES.PENALTY_CYCLES > DSB_FILL.EXCEED_DSB_LINES > ITLB.ITLB_FLUSH > OFFCORE_REQUESTS.DEMAND_DATA_RD > OFFCORE_REQUESTS.DEMAND_CODE_RD > OFFCORE_REQUESTS.DEMAND_RFO > OFFCORE_REQUESTS.ALL_DATA_RD > UOPS_EXECUTED.THREAD > UOPS_EXECUTED.CORE > OFF_CORE_RESPONSE_0 > OFF_CORE_RESPONSE_1 > TLB_FLUSH.DTLB_THREAD > TLB_FLUSH.STLB_ANY > INST_RETIRED.ANY_P > INST_RETIRED.ALL > OTHER_ASSISTS.AVX_STORE > OTHER_ASSISTS.AVX_TO_SSE > OTHER_ASSISTS.SSE_TO_AVX > UOPS_RETIRED.ALL > UOPS_RETIRED.RETIRE_SLOTS > MACHINE_CLEARS.MEMORY_ORDERING > MACHINE_CLEARS.SMC > MACHINE_CLEARS.MASKMOV > BR_INST_RETIRED.ALL_BRANCHES > BR_INST_RETIRED.CONDITIONAL > BR_INST_RETIRED.NEAR_CALL > BR_INST_RETIRED.ALL_BRANCHES > BR_INST_RETIRED.NEAR_RETURN > BR_INST_RETIRED.NOT_TAKEN > BR_INST_RETIRED.NEAR_TAKEN > BR_INST_RETIRED.FAR_BRANCH > BR_MISP_RETIRED.ALL_BRANCHES > BR_MISP_RETIRED.CONDITIONAL > BR_MISP_RETIRED.NEAR_CALL > BR_MISP_RETIRED.ALL_BRANCHES > BR_MISP_RETIRED.NOT_TAKEN > BR_MISP_RETIRED.TAKEN > FP_ASSIST.X87_OUTPUT > FP_ASSIST.X87_INPUT > FP_ASSIST.SIMD_OUTPUT > FP_ASSIST.SIMD_INPUT > FP_ASSIST.ANY > ROB_MISC_EVENTS.LBR_INSERTS > MEM_TRANS_RETIRED.LOAD_LATENCY > MEM_TRANS_RETIRED.PRECISE_STORE > MEM_UOP_RETIRED.LOADS > MEM_UOP_RETIRED.STORES > MEM_UOP_RETIRED.STLB_MISS > MEM_UOP_RETIRED.LOCK > MEM_UOP_RETIRED.SPLIT > MEM_UOP_RETIRED.ALL > MEM_LOAD_UOPS_RETIRED.L1_HIT > MEM_LOAD_UOPS_RETIRED.L2_HIT > MEM_LOAD_UOPS_RETIRED.LLC_HIT > MEM_LOAD_UOPS_RETIRED.HIT_LFB > MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_MISS > MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT > MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HITM > MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_NONE > MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM > L2_TRANS.DEMAND_DATA_RD > L2_TRANS.RFO > L2_TRANS.CODE_RD > L2_TRANS.ALL_PF > L2_TRANS.L1D_WB > L2_TRANS.L2_FILL > L2_TRANS.L2_WB > L2_TRANS.ALL_REQUESTS > L2_LINES_IN.I > L2_LINES_IN.S > L2_LINES_IN.E > L2_LINES_IN.ALL > L2_LINES_OUT.DEMAND_CLEAN > L2_LINES_OUT.DEMAND_DIRTY > L2_LINES_OUT.PF_CLEAN > L2_LINES_OUT.PF_DIRTY > IAF > INSTR_RETIRED_ANY > CPU_CLK_UNHALTED_CORE > CPU_CLK_UNHALTED_REF From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 08:43:16 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F46C106566C for ; Thu, 6 Sep 2012 08:43:16 +0000 (UTC) (envelope-from rdivacky@vlakno.cz) Received: from vlakno.cz (lev.vlakno.cz [46.28.110.116]) by mx1.freebsd.org (Postfix) with ESMTP id 854C68FC0C for ; Thu, 6 Sep 2012 08:43:15 +0000 (UTC) Received: by vlakno.cz (Postfix, from userid 1002) id CB2EC7F38AE; Thu, 6 Sep 2012 10:43:12 +0200 (CEST) Date: Thu, 6 Sep 2012 10:43:12 +0200 From: Roman Divacky To: Steve Kargl Message-ID: <20120906084312.GA13223@freebsd.org> References: <5046670C.6050500@andric.com> <20120904214344.GA17723@troutmask.apl.washington.edu> <504679CB.90204@andric.com> <20120904221413.GA19395@troutmask.apl.washington.edu> <50471BEE.6030708@andric.com> <20120905221310.GA97847@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120905221310.GA97847@troutmask.apl.washington.edu> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Garrett Cooper , Dimitry Andric , freebsd-current@freebsd.org Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 08:43:16 -0000 On Wed, Sep 05, 2012 at 03:13:11PM -0700, Steve Kargl wrote: > On Wed, Sep 05, 2012 at 11:31:26AM +0200, Dimitry Andric wrote: > > On 2012-09-05 01:40, Garrett Cooper wrote: > > ... > > > Steve does have a point. Posting the results of > > >CFLAGS/CPPFLAGS/LDFLAGS/etc for config.log (and maybe poking through > > >the code to figure out what *FLAGS were used elsewhere) is more > > >valuable than the data is in its current state (unfortunately.. > > >autoconf makes things more complicated). > > > > 1) For building the FreeBSD in-tree version of clang 3.2: > > > > -O2 -pipe -fno-strict-aliasing > > > > 2) For building the FreeBSD in-tree version of gcc 4.2.1: > > > > -O2 -pipe > > > > 3) For building Boost 1.50.0: > > > > -ftemplate-depth-128 -O3 -finline-functions > > > > Dimitry thanks for the follow-up. I performed an unscientific > (micro)benchmark of /usr/bin/cc vs /usr/bin/clang where cc is > the base system's gcc 4.2.1. Here's what I found/feared. > > Compiling libm on > > CPU: AMD Opteron(tm) Processor 248 (2192.01-MHz K8-class CPU) > Origin = "AuthenticAMD" Id = 0xf5a Family = f Model = 5 Stepping = 10 > Features=0x78bfbff MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2> > AMD Features=0xe0500800 > > with default CFLAGS (ie., -O2 -pipe) and -march=opteron. Was this compiled as amd64 or i386? Also, can you send me the test case? So that we can explore the difference. The working theory now is SSE vs FPU mathematics, but it would be nice to see the testcase. Thank you, roman > Using 'setenv CC /usr/bin/cc' with 3 runs of > > make clean > time make -DNO_MAN > > yields > > 69.39 real 52.00 user 38.55 sys > 69.57 real 52.35 user 38.37 sys > 69.48 real 52.25 user 38.38 sys > > Now, repeating with 'setenv CC /usr/bin/clang' yields > > 39.65 real 21.86 user 17.37 sys > 40.91 real 21.48 user 17.91 sys > 39.77 real 21.65 user 17.64 sys > > So, clang does appear to be faster in this particular > compiling speed benchmark. > > However, if I know build my test program for libm's j0f() > function where the only difference is whether libm was > built with /usr/bin/cc or /usr/bin/clang, I observe the > following results. > > 1234567 x values in the interval [0:25] > > gcc libm | clang libm > ----------------|----------------- > ULP <= 0.6 --> 565515 (45.81%) | 513763 (41.61%) > 0.6 < ULP <= 0.7 --> 74148 ( 6.01%) | 67221 ( 5.44%) > 0.7 < ULP <= 0.8 --> 69112 ( 5.60%) | 62846 ( 5.09%) > 0.8 < ULP <= 0.9 --> 63798 ( 5.17%) | 58217 ( 4.72%) > 0.9 < ULP <= 1.0 --> 58679 ( 4.75%) | 53834 ( 4.36%) > 1.0 < ULP <= 2.0 --> 328221 (26.59%) | 306728 (24.84%) > 2.0 < ULP <= 3.0 --> 65323 ( 5.29%) | 63452 ( 5.14%) > 3.0 < ULP --> 9771 ( 0.79%) | 108506 ( 8.79%) > > gcc libm | clang libm > -----------------------|-------------------- > MAX ULP: 12152.27637 | 1129606938624.00000 > x at MAX ULP: 5.520077 0x1.6148f2p+2 | 2.404833 0x1.33d19p+1 > > Speed test with gcc libm. > 1234567 j0f calls in 0.193427 seconds. > 1234567 j0f calls in 0.193410 seconds. > 1234567 j0f calls in 0.194158 seconds. > > Speed test with clang libm. > 1234567 j0f calls in 0.180260 seconds. > 1234567 j0f calls in 0.180130 seconds. > 1234567 j0f calls in 0.179739 seconds. > > So, although the clang built j0f() appears to be faster than > the gcc built j0f(), the clang built j0f() has much worse > accuracy issues. > > -- > Steve > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 10:20:29 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 778B01065670; Thu, 6 Sep 2012 10:20:29 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from theravensnest.org (theraven.freebsd.your.org [216.14.102.27]) by mx1.freebsd.org (Postfix) with ESMTP id 3DB8A8FC0C; Thu, 6 Sep 2012 10:20:29 +0000 (UTC) Received: from c120.sec.cl.cam.ac.uk (c120.sec.cl.cam.ac.uk [128.232.18.120]) (authenticated bits=0) by theravensnest.org (8.14.5/8.14.5) with ESMTP id q86AKMnw094546 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Thu, 6 Sep 2012 10:20:23 GMT (envelope-from theraven@FreeBSD.org) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=us-ascii From: David Chisnall In-Reply-To: <20120906084312.GA13223@freebsd.org> Date: Thu, 6 Sep 2012 11:20:24 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <122361C5-7B83-494B-A580-970FAFF83CAA@FreeBSD.org> References: <5046670C.6050500@andric.com> <20120904214344.GA17723@troutmask.apl.washington.edu> <504679CB.90204@andric.com> <20120904221413.GA19395@troutmask.apl.washington.edu> <50471BEE.6030708@andric.com> <20120905221310.GA97847@troutmask.apl.washington.edu> <20120906084312.GA13223@freebsd.org> To: Roman Divacky X-Mailer: Apple Mail (2.1278) Cc: Garrett Cooper , Dimitry Andric , freebsd-current@FreeBSD.org, Steve Kargl Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 10:20:29 -0000 On 6 Sep 2012, at 09:43, Roman Divacky wrote: > Was this compiled as amd64 or i386? Also, can you send me the test = case? > So that we can explore the difference. The working theory now is SSE = vs FPU > mathematics, but it would be nice to see the testcase. There may also be a difference in whether -ffast-math is the default on = each compiler. On x86, this will replace a number of libm calls with = (much faster, but less accurate) SSE or x87 instructions. If this is = enabled by default with clang and not with gcc, it would account for the = difference. =20 David= From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 10:37:47 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 144041065672 for ; Thu, 6 Sep 2012 10:37:47 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from mail.ultra-secure.de (mail.ultra-secure.de [78.47.114.122]) by mx1.freebsd.org (Postfix) with ESMTP id 6AA688FC0C for ; Thu, 6 Sep 2012 10:37:45 +0000 (UTC) Received: (qmail 25032 invoked by uid 89); 6 Sep 2012 10:37:45 -0000 Received: by simscan 1.4.0 ppid: 25027, pid: 25029, t: 0.0354s scanners: attach: 1.4.0 clamav: 0.97.3/m:54/d:15319 Received: from unknown (HELO suse2.ip-tech.ch) (rainer@ultra-secure.de@212.71.117.1) by mail.ultra-secure.de with ESMTPA; 6 Sep 2012 10:37:45 -0000 Date: Thu, 6 Sep 2012 12:37:44 +0200 From: Rainer Duffner To: freebsd-current@FreeBSD.org Message-ID: <20120906123744.3883e075@suse2.ip-tech.ch> X-Mailer: Claws Mail 3.7.10 (GTK+ 2.22.1; x86_64-unknown-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Subject: Where to ask questions about poudriere? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 10:37:47 -0000 Hi, I'm trying to get poudriere working with the following settings: f2d169d8-20d2-41d4-8e43-8a9fc5a2b509# cat /usr/local/etc/poudriere.conf |grep -v ^# |grep -v ^$ ZPOOL=datapool FTPHOST=ftp.ch.freebsd.org FREEBSD_HOST=http://ftp.ch.freebsd.org/ RESOLV_CONF=/etc/resolv.conf BASEFS=/usr/local/poudriere USE_PORTLINT=no USE_TMPFS=yes DISTFILES_CACHE=/usr/ports/distfiles CSUP_HOST=localhost CHECK_CHANGED_OPTIONS=yes PKG_REPO_SIGNING_KEY=/etc/ssl/keys/repo.bla.ch.key CCACHE_DIR=/data/cache/ccache f2d169d8-20d2-41d4-8e43-8a9fc5a2b509# cat /usr/local/etc/poudriere.d/make.conf WITH_CCACHE_BUILD=yes USE_LOCAL_MK=yes f2d169d8-20d2-41d4-8e43-8a9fc5a2b509# poudriere bulk -f ~/portlist-current-php53-mysql55 -j 90amd64 ====>> Mounting system devices for 90amd64 /etc/resolv.conf -> /usr/local/poudriere/jails/90amd64/etc/resolv.conf ====>> Starting jail 90amd64 ====>> Mounting ports filesystems for 90amd64 ====>> Mounting ccache from /data/cache/ccache mount: /usr/local/poudriere/jails/90amd64/usr/ports/distfiles: No such file or directory ====>> Umounting file systems Failed to mount the distfile directory You have new mail. f2d169d8-20d2-41d4-8e43-8a9fc5a2b509# zfs list NAME USED AVAIL REFER MOUNTPOINT datapool 1.51G 47.5G 32K /datapool datapool/poudriere 1.51G 47.5G 33K /datapool/poudriere datapool/poudriere/data 36K 47.5G 36K /usr/local/poudriere/data datapool/poudriere/jails 1017M 47.5G 31K /datapool/poudriere/jails datapool/poudriere/jails/90amd64 1017M 47.5G 1017M /usr/local/poudriere/jails/90amd64 datapool/poudriere/ports 534M 47.5G 31K /datapool/poudriere/ports datapool/poudriere/ports/current 534M 47.5G 534M /usr/local/poudriere/ports/current What is the problem? From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 10:37:43 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 571C6106564A; Thu, 6 Sep 2012 10:37:43 +0000 (UTC) (envelope-from ohartman@mail.zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id ED4E78FC0A; Thu, 6 Sep 2012 10:37:42 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.69) with esmtp (envelope-from ) id <1T9ZSp-00051K-Vk>; Thu, 06 Sep 2012 12:37:36 +0200 Received: from telesto.geoinf.fu-berlin.de ([130.133.86.198]) by inpost2.zedat.fu-berlin.de (Exim 4.69) with esmtpsa (envelope-from ) id <1T9ZSp-0005gk-S0>; Thu, 06 Sep 2012 12:37:35 +0200 Message-ID: <50487CEB.9030001@mail.zedat.fu-berlin.de> Date: Thu, 06 Sep 2012 12:37:31 +0200 From: "O. Hartmann" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120810 Thunderbird/14.0 MIME-Version: 1.0 To: Matthieu Volat References: <5047659D.8000107@mail.zedat.fu-berlin.de> <20120906121639.4b7d894d089b2ddeb42f31b4@alkumuna.eu> In-Reply-To: <20120906121639.4b7d894d089b2ddeb42f31b4@alkumuna.eu> X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigF4D6AA5E9A851DFD2901DADB" X-Originating-IP: 130.133.86.198 X-Mailman-Approved-At: Thu, 06 Sep 2012 11:29:18 +0000 Cc: Current FreeBSD , freebsd-ports@freebsd.org Subject: Re: Help. Porting "FreeOCL" fails (atomic_ops.h missing, CLANG++ libc++ issues ...) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 10:37:43 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigF4D6AA5E9A851DFD2901DADB Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 09/06/12 12:16, Matthieu Volat wrote: > On Wed, 05 Sep 2012 16:45:49 +0200 > "O. Hartmann" wrote: >=20 >> Hello. >> >> FreeBSD has fallen back far behind the standards of modern scientific >> computing and I dsperately look for solutions having OpenCL support on= >> FreeBSD anyway. >> >> I stumbled into this project recently: >> >> FreeOCL at >> http://code.google.com/p/freeocl/ >> >> or the sources located at >> >> http://code.google.com/p/freeocl/downloads/detail?name=3DFreeOCL-0.3.6= -src.tar.gz&can=3D2&q=3D >> >> For your convenience, please find my naive attempt of a port attached = to >> this email. >> >> Well, I tried LLVM/CLANG, but Cmake of the sources fairly fails many >> checks especuially for OpenMP. Using clang++ requisites the usage of t= he >> new libc++ (CXXFLAGS+=3D -stdlib=3Dlibc++). But this fails with this e= rror: >> >> >> [...] >> [ 17%] Building CXX object src/CMakeFiles/FreeOCL.dir/codebuilder.cpp.= o >> In file included from >> /usr/ports/devel/freeocl/work/FreeOCL-0.3.6-Source/src/codebuilder.cpp= :26: >> /usr/ports/devel/freeocl/work/FreeOCL-0.3.6-Source/src/parser/parser.h= :118:15: >> error: no viable conversion from 'std::__1::basic_istream' to >> 'const bool' >> const bool ok =3D in.get(c); >> ^ ~~~~~~~~~ >> /usr/ports/devel/freeocl/work/FreeOCL-0.3.6-Source/src/codebuilder.cpp= :444:13: >> warning: 70 enumeration values not handled in switch: 'VOID', 'BOOL', >> 'HALF'... [-Wswitch] >> switch(native->get_type_id()) >> ^ >> 1 warning and 1 error generated. >> *** [src/CMakeFiles/FreeOCL.dir/codebuilder.cpp.o] Error code 1 >> [..] >> >> >> I tried also setting USE_GCC=3D 4.7+ using gcc-4.7 I installed, but th= at >> fails with >> >> [...] >> -- Build files have been written to: >> /usr/ports/devel/freeocl/work/FreeOCL-0.3.6-Source >> =3D=3D=3D> Building for freeocl-0.3.6 >> Scanning dependencies of target FreeOCL >> [ 1%] Building CXX object src/CMakeFiles/FreeOCL.dir/freeocl.cpp.o >> In file included from >> /usr/ports/devel/freeocl/work/FreeOCL-0.3.6-Source/src/freeocl.cpp:18:= 0: >> /usr/ports/devel/freeocl/work/FreeOCL-0.3.6-Source/src/freeocl.h:71:2:= >> error: 'u_int64_t' does not name a type >> *** [src/CMakeFiles/FreeOCL.dir/freeocl.cpp.o] Error code 1 >> [...] >> >> I think patches are required. >> >> More disturbing is the use of gcc-4.6 via >> >> USE_GCC=3D 4.6+ >> >> The error is then a compalin about a missing include file >> >> atomic_ops.h >> >> which is located in the abandoned KSE facility of the kernel, so the >> includes seem not to be installed anymore - but since I'm not involved= >> in the development, I can only guess and ask the people here. >> >> Maybe some experienced freeBSD developers with the same need for OpenC= l >> is willing to pick up this and help a bit. >> >> Well, as far as I see, the FreeOCL project of Roland Borchard is a >> OpenCL library for CPU usage. As far as I can understand, if we on >> FreeBSd could have a library "libOpenCL" as this is usually installed = by >> the nVidia BLOB drivers on Linux for the use of OpenCL/GPGPU support >> with their GPUs, this would fill a still painfull open gap in FreeBSD = as >> a scientific development platform. >> >> There is still no suitable compiler for OpenCL out here for freeBSD, b= ut >> I have still the hope that LLVM can provide such a thing in the near f= uture. >> >> I cross post this posting also to "performance" in the hope finding so= me >> people attracted and "lurd" into this subject. >> >> Regards, >> >> O. Hartmann >=20 > Hello, >=20 > I've taken a quick look in that implementation : it seems to depends on= devel/libatomic_ops even if the devs did not write any CMake checks for = that (grrr). >=20 > After installing it, I was able to build the port with gcc-46 without p= roblems, if I find some time, I will try to test it later. >=20 > Btw, did you had a look to poCL too (http://sourceforge.net/apps/mediaw= iki/pocl/index.php?title=3DMain_Page)? I know the Debian people were cons= idering it for a base implementation, but I'm a bit clueless about all th= ose foss versions coming. A comparison could be helpful ^^ >=20 Hello Matthieu. Thanks for the hint for that port. I will have a look at poCL. Thanks for that hint. I cross post to "current", maybe those aspects have influence of some add ons in CURRENT in the future. Oliver --------------enigF4D6AA5E9A851DFD2901DADB Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBAgAGBQJQSHzvAAoJEOgBcD7A/5N8sr4H/2Upoq8nbFYC/U61p1qSn8uc ERhcQl1xeYzCb5qsRCEENS+q07+klgC4Cj+HrfYpXPjCZNQ7fpizs1OQa7Yjk+u8 yQTvazv3WdAWXSIW0x2kh55tK+qPA9N2NOEzyVNk4SzrwdZBubK0iIBROcMutJ5j LcaqxaAW4Lib5eQkOSyIP/eP5zWpe3Z/HR/7+HIcqLzdA9cEsbebmKk4j03NyI5V voiOkfPLbLYGFFcZsQeeFecMfxyrrqZxkudxNmrJiDI8achx3VwxCarZQ/MI4s7i tF7DTrO2JpXiRQEjv6HkeRgEJ4I9KCDDN8VjhwoVd/hO4xFQDHiXx0LNMWbxUJw= =hpRT -----END PGP SIGNATURE----- --------------enigF4D6AA5E9A851DFD2901DADB-- From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 11:14:57 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 883A5106566B; Thu, 6 Sep 2012 11:14:57 +0000 (UTC) (envelope-from ohartman@mail.zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id 2AC4A8FC08; Thu, 6 Sep 2012 11:14:56 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.69) with esmtp (envelope-from ) id <1T9a2x-000543-CB>; Thu, 06 Sep 2012 13:14:55 +0200 Received: from telesto.geoinf.fu-berlin.de ([130.133.86.198]) by inpost2.zedat.fu-berlin.de (Exim 4.69) with esmtpsa (envelope-from ) id <1T9a2x-0008Dh-8W>; Thu, 06 Sep 2012 13:14:55 +0200 Message-ID: <504885AA.1000300@mail.zedat.fu-berlin.de> Date: Thu, 06 Sep 2012 13:14:50 +0200 From: "O. Hartmann" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120810 Thunderbird/14.0 MIME-Version: 1.0 To: Matthieu Volat References: <5047659D.8000107@mail.zedat.fu-berlin.de> <20120906121639.4b7d894d089b2ddeb42f31b4@alkumuna.eu> In-Reply-To: <20120906121639.4b7d894d089b2ddeb42f31b4@alkumuna.eu> X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig36C883B3731B9F0E75AEB96A" X-Originating-IP: 130.133.86.198 X-Mailman-Approved-At: Thu, 06 Sep 2012 11:29:37 +0000 Cc: "freebsd >> Current FreeBSD" , freebsd-ports@freebsd.org Subject: Re: Help. Porting "FreeOCL" fails (atomic_ops.h missing, CLANG++ libc++ issues ...) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 11:14:57 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig36C883B3731B9F0E75AEB96A Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 09/06/12 12:16, Matthieu Volat wrote: > On Wed, 05 Sep 2012 16:45:49 +0200 > "O. Hartmann" wrote: >=20 >> Hello. >> >> FreeBSD has fallen back far behind the standards of modern scientific >> computing and I dsperately look for solutions having OpenCL support on= >> FreeBSD anyway. >> >> I stumbled into this project recently: >> >> FreeOCL at >> http://code.google.com/p/freeocl/ >> >> or the sources located at >> >> http://code.google.com/p/freeocl/downloads/detail?name=3DFreeOCL-0.3.6= -src.tar.gz&can=3D2&q=3D >> >> For your convenience, please find my naive attempt of a port attached = to >> this email. >> >> Well, I tried LLVM/CLANG, but Cmake of the sources fairly fails many >> checks especuially for OpenMP. Using clang++ requisites the usage of t= he >> new libc++ (CXXFLAGS+=3D -stdlib=3Dlibc++). But this fails with this e= rror: >> >> >> [...] >> [ 17%] Building CXX object src/CMakeFiles/FreeOCL.dir/codebuilder.cpp.= o >> In file included from >> /usr/ports/devel/freeocl/work/FreeOCL-0.3.6-Source/src/codebuilder.cpp= :26: >> /usr/ports/devel/freeocl/work/FreeOCL-0.3.6-Source/src/parser/parser.h= :118:15: >> error: no viable conversion from 'std::__1::basic_istream' to >> 'const bool' >> const bool ok =3D in.get(c); >> ^ ~~~~~~~~~ >> /usr/ports/devel/freeocl/work/FreeOCL-0.3.6-Source/src/codebuilder.cpp= :444:13: >> warning: 70 enumeration values not handled in switch: 'VOID', 'BOOL', >> 'HALF'... [-Wswitch] >> switch(native->get_type_id()) >> ^ >> 1 warning and 1 error generated. >> *** [src/CMakeFiles/FreeOCL.dir/codebuilder.cpp.o] Error code 1 >> [..] >> >> >> I tried also setting USE_GCC=3D 4.7+ using gcc-4.7 I installed, but th= at >> fails with >> >> [...] >> -- Build files have been written to: >> /usr/ports/devel/freeocl/work/FreeOCL-0.3.6-Source >> =3D=3D=3D> Building for freeocl-0.3.6 >> Scanning dependencies of target FreeOCL >> [ 1%] Building CXX object src/CMakeFiles/FreeOCL.dir/freeocl.cpp.o >> In file included from >> /usr/ports/devel/freeocl/work/FreeOCL-0.3.6-Source/src/freeocl.cpp:18:= 0: >> /usr/ports/devel/freeocl/work/FreeOCL-0.3.6-Source/src/freeocl.h:71:2:= >> error: 'u_int64_t' does not name a type >> *** [src/CMakeFiles/FreeOCL.dir/freeocl.cpp.o] Error code 1 >> [...] >> >> I think patches are required. >> >> More disturbing is the use of gcc-4.6 via >> >> USE_GCC=3D 4.6+ >> >> The error is then a compalin about a missing include file >> >> atomic_ops.h >> >> which is located in the abandoned KSE facility of the kernel, so the >> includes seem not to be installed anymore - but since I'm not involved= >> in the development, I can only guess and ask the people here. >> >> Maybe some experienced freeBSD developers with the same need for OpenC= l >> is willing to pick up this and help a bit. >> >> Well, as far as I see, the FreeOCL project of Roland Borchard is a >> OpenCL library for CPU usage. As far as I can understand, if we on >> FreeBSd could have a library "libOpenCL" as this is usually installed = by >> the nVidia BLOB drivers on Linux for the use of OpenCL/GPGPU support >> with their GPUs, this would fill a still painfull open gap in FreeBSD = as >> a scientific development platform. >> >> There is still no suitable compiler for OpenCL out here for freeBSD, b= ut >> I have still the hope that LLVM can provide such a thing in the near f= uture. >> >> I cross post this posting also to "performance" in the hope finding so= me >> people attracted and "lurd" into this subject. >> >> Regards, >> >> O. Hartmann >=20 > Hello, >=20 > I've taken a quick look in that implementation : it seems to depends on= devel/libatomic_ops even if the devs did not write any CMake checks for = that (grrr). >=20 > After installing it, I was able to build the port with gcc-46 without p= roblems, if I find some time, I will try to test it later. >=20 > Btw, did you had a look to poCL too (http://sourceforge.net/apps/mediaw= iki/pocl/index.php?title=3DMain_Page)? I know the Debian people were cons= idering it for a base implementation, but I'm a bit clueless about all th= ose foss versions coming. A comparison could be helpful ^^ >=20 I tried to add RUN_DEPENDS=3D ${LOCALBASE}/lib/libatomic_ops.a:${PORTSDIR}/devel/libatomic_ops to my provided Makefile, but this doesn't install the port devel/libatomic_ops. This is weird and inconsistent. I follow exact the steps suggested in the Porter's handbook, the _DEPENDS=3D section. The above RUN_DEPENDS=3D = tag should ensure a check for the existence of the static library /usr/local/lib/libatomic_ops.a and if not existent, then install it. It doesn't work. Unfortunately, LIB_DEPENDS is considered for "shared libraries", so it isn't suitable. But LIB_DEPENDS get recognized, even if it fails, while RUN_DEPENDS seems not to be touched by the build process anyway ... --------------enig36C883B3731B9F0E75AEB96A Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBAgAGBQJQSIWvAAoJEOgBcD7A/5N8r90H/3kWR9sJ+qQ7mGRhUuehDjJS gtpIxE441nJeS/vCj45Pf+1ZyD8QSkGt7VYn0VI3BVBGIe54FY5Wblri3NZ6+BK2 YAofAxLZY2nvI52Gb6CBrJ2WiHAGZ+FTHgcFJrGk8LBzyHMfEpzhhf2nk/z97K46 kUUAOTR5Vex1XGjrbjAQ3JbOvPhdQlcWgL2UtCeZ34S42Cw9J0nQU3MoitcCZOc2 HLpTXoUxhOuTaWRUzjW3BySLyiq/mdd/sOHNBvmpcj0ffFASpCbnyCjwsvfxgbsc U5s37+hFnIg2CJ7/zAgTPYW8BaiHyyOWs4gHODyXulFSRTw9UhROa6KVgadLB28= =yMvA -----END PGP SIGNATURE----- --------------enig36C883B3731B9F0E75AEB96A-- From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 11:59:03 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D31D8106566B for ; Thu, 6 Sep 2012 11:59:03 +0000 (UTC) (envelope-from yerenkow@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9780D8FC19 for ; Thu, 6 Sep 2012 11:59:03 +0000 (UTC) Received: by iayy25 with SMTP id y25so2457633iay.13 for ; Thu, 06 Sep 2012 04:59:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=b6mNPPz4DXDIECxFGcgIm+EgperRfCidSKBv37j1Hc4=; b=HlX+6ou8uWG1YkCWj806EjsK4oCNzp7kQgFCL4tUtPlsc5q1W7vIQN7SNRrVlqt+er hA/rJ6SHcNLKfDRxWoq5culO4Ci/rbkt0pK3DwpHl+k0nxfXmqq+rCEIPvIjIBC3Y1NC Dcl+trqfjZFIOwQlJrbbSY5NpGyFQxqFdtBUUqul7ueZPXGrBXt6+ZsB38kVkuyq3vAD fw52yGVpzuQki9yFZoNM9AKPB6BekOzfSG6JMK7VNzq5pGO7W8LvRKk/zCeuUKKq1gDJ bQVU/05yEn9lY9O12JlyVb0e04a+uDmU4LgBAQrcumZh6R/LBKiNg0jmuepwD4Bctx3E X47Q== MIME-Version: 1.0 Received: by 10.50.94.197 with SMTP id de5mr2495058igb.32.1346932743061; Thu, 06 Sep 2012 04:59:03 -0700 (PDT) Received: by 10.64.28.36 with HTTP; Thu, 6 Sep 2012 04:59:03 -0700 (PDT) In-Reply-To: <20120906123744.3883e075@suse2.ip-tech.ch> References: <20120906123744.3883e075@suse2.ip-tech.ch> Date: Thu, 6 Sep 2012 14:59:03 +0300 Message-ID: From: Alexander Yerenkow To: Rainer Duffner Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-current@freebsd.org Subject: Re: Where to ask questions about poudriere? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 11:59:03 -0000 I don't see where you specify your zfs tank, is it missed in mail, or in conf? 2012/9/6 Rainer Duffner > Hi, > > I'm trying to get poudriere working with the following settings: > > f2d169d8-20d2-41d4-8e43-8a9fc5a2b509# cat /usr/local/etc/poudriere.conf > |grep -v ^# |grep -v ^$ ZPOOL=datapool > FTPHOST=ftp.ch.freebsd.org > FREEBSD_HOST=http://ftp.ch.freebsd.org/ > RESOLV_CONF=/etc/resolv.conf > BASEFS=/usr/local/poudriere > USE_PORTLINT=no > USE_TMPFS=yes > DISTFILES_CACHE=/usr/ports/distfiles > CSUP_HOST=localhost > CHECK_CHANGED_OPTIONS=yes > PKG_REPO_SIGNING_KEY=/etc/ssl/keys/repo.bla.ch.key > CCACHE_DIR=/data/cache/ccache > > > f2d169d8-20d2-41d4-8e43-8a9fc5a2b509# > cat /usr/local/etc/poudriere.d/make.conf WITH_CCACHE_BUILD=yes > USE_LOCAL_MK=yes > > f2d169d8-20d2-41d4-8e43-8a9fc5a2b509# poudriere bulk -f > ~/portlist-current-php53-mysql55 -j 90amd64 ====>> Mounting system > devices for 90amd64 /etc/resolv.conf > -> /usr/local/poudriere/jails/90amd64/etc/resolv.conf ====>> Starting > jail 90amd64 ====>> Mounting ports filesystems for 90amd64 > ====>> Mounting ccache from /data/cache/ccache > mount: /usr/local/poudriere/jails/90amd64/usr/ports/distfiles: No such > file or directory ====>> Umounting file systems > Failed to mount the distfile directory > You have new mail. > > > f2d169d8-20d2-41d4-8e43-8a9fc5a2b509# zfs list > NAME USED AVAIL REFER MOUNTPOINT > datapool 1.51G 47.5G 32K /datapool > datapool/poudriere 1.51G 47.5G > 33K /datapool/poudriere datapool/poudriere/data 36K > 47.5G 36K /usr/local/poudriere/data > datapool/poudriere/jails 1017M 47.5G > 31K /datapool/poudriere/jails datapool/poudriere/jails/90amd64 1017M > 47.5G 1017M /usr/local/poudriere/jails/90amd64 > datapool/poudriere/ports 534M 47.5G > 31K /datapool/poudriere/ports datapool/poudriere/ports/current 534M > 47.5G 534M /usr/local/poudriere/ports/current > > > What is the problem? > > > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > -- Regards, Alexander Yerenkow From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 12:19:42 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3BD46106566B; Thu, 6 Sep 2012 12:19:42 +0000 (UTC) (envelope-from zeising@daemonic.se) Received: from mail.lysator.liu.se (mail.lysator.liu.se [IPv6:2001:6b0:17:f0a0::3]) by mx1.freebsd.org (Postfix) with ESMTP id A90788FC15; Thu, 6 Sep 2012 12:19:41 +0000 (UTC) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id 7C71A40020; Thu, 6 Sep 2012 14:19:40 +0200 (CEST) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id 716E840021; Thu, 6 Sep 2012 14:19:40 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bernadotte.lysator.liu.se X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=AWL autolearn=disabled version=3.3.1 X-Spam-Score: 0.0 Received: from mx.daemonic.se (mx.daemonic.se [IPv6:2001:470:dca9:0:1::3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id E2A6740020; Thu, 6 Sep 2012 14:19:37 +0200 (CEST) Received: from mailscanner.daemonic.se (mailscanner.daemonic.se [IPv6:2001:470:dca9:0:1::6]) by mx.daemonic.se (Postfix) with ESMTPS id 3XCLST49xZz8gtM; Thu, 6 Sep 2012 14:19:37 +0200 (CEST) X-Virus-Scanned: amavisd-new at daemonic.se Received: from mx.daemonic.se ([IPv6:2001:470:dca9:0:1::3]) (using TLS with cipher CAMELLIA256-SHA) by mailscanner.daemonic.se (mailscanner.daemonic.se [2001:470:dca9:0:1::6]) (amavisd-new, port 10025) with ESMTPS id lqrpE1tQCDaU; Thu, 6 Sep 2012 14:19:32 +0200 (CEST) Received: from mail.daemonic.se (mail.daemonic.se [10.1.0.4]) by mx.daemonic.se (Postfix) with ESMTPS id 3XCLSN0kZwz8gtL; Thu, 6 Sep 2012 14:19:32 +0200 (CEST) Received: from [10.32.0.4] (vivi.daemonic.se [10.32.0.4]) by mail.daemonic.se (Postfix) with ESMTPSA id 3XCLSM4mHnz9Ctj; Thu, 6 Sep 2012 14:19:31 +0200 (CEST) Message-ID: <504894D8.6020007@daemonic.se> Date: Thu, 06 Sep 2012 14:19:36 +0200 From: Niclas Zeising User-Agent: Mutt/1.5.21 MIME-Version: 1.0 To: Dimitry Andric References: <5047093A.5060408@zedat.fu-berlin.de> <50471757.4070202@FreeBSD.org> In-Reply-To: <50471757.4070202@FreeBSD.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV using ClamSMTP X-Mailman-Approved-At: Thu, 06 Sep 2012 12:40:15 +0000 Cc: Current FreeBSD , "O. Hartmann" Subject: Re: Firefox-15/Thunderbird-15: won't compile on FreeBSD 10.0-CUR: /jsproxy.h:17:7: error: visibility does not match previous declaration X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 12:19:42 -0000 On 2012-09-05 11:11, Dimitry Andric wrote: > On 2012-09-05 10:11, O. Hartmann wrote: >> Udating/reinstalling of both ports www/firefox (15.0) and >> mail/thunderbird (15) fail with an error like showed below. > .... >> ./jsproxy.h:17:7: error: visibility does not match previous declaration >> class JS_FRIEND_API(BaseProxyHandler) { > > Please see: http://llvm.org/bugs/show_bug.cgi?id=13688 > > This is either a bug in clang 3.2, or alternatively, a problem with > inconsistent declarations in libc++. Except that libc++ is not used here, or at least I can't see a -stdlib=libc++ in the command line string, so it is probably not related to libc++. Regards! -- Niclas From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 12:57:19 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5DBDB106566B; Thu, 6 Sep 2012 12:57:19 +0000 (UTC) (envelope-from dimitry@andric.com) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id 110388FC08; Thu, 6 Sep 2012 12:57:19 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:f8a2:c245:f52d:c435] (unknown [IPv6:2001:7b8:3a7:0:f8a2:c245:f52d:c435]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 541A65C37; Thu, 6 Sep 2012 14:57:18 +0200 (CEST) Message-ID: <50489DB1.4070102@andric.com> Date: Thu, 06 Sep 2012 14:57:21 +0200 From: Dimitry Andric User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20120828 Thunderbird/16.0 MIME-Version: 1.0 To: David Chisnall References: <5046670C.6050500@andric.com> <20120904214344.GA17723@troutmask.apl.washington.edu> <504679CB.90204@andric.com> <20120904221413.GA19395@troutmask.apl.washington.edu> <50471BEE.6030708@andric.com> <20120905221310.GA97847@troutmask.apl.washington.edu> <20120906084312.GA13223@freebsd.org> <122361C5-7B83-494B-A580-970FAFF83CAA@FreeBSD.org> In-Reply-To: <122361C5-7B83-494B-A580-970FAFF83CAA@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Garrett Cooper , Roman Divacky , freebsd-current@FreeBSD.org, Steve Kargl Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 12:57:19 -0000 On 2012-09-06 12:20, David Chisnall wrote: ... > There may also be a difference in whether -ffast-math is the default on each compiler. On x86, this will replace a number of libm calls with (much faster, but less accurate) SSE or x87 instructions. If this is enabled by default with clang and not with gcc, it would account for the difference. No, -ffast-math is not enabled by default in clang, as far as I can tell. Also, the help text for the option says: "Enable the *frontend*'s 'fast-math' mode. This has no effect on optimizations, but provides a preprocessor macro __FAST_MATH__ the same as GCC's -ffast-math flag." From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 13:35:13 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3E936106566B; Thu, 6 Sep 2012 13:35:13 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 924CF8FC14; Thu, 6 Sep 2012 13:35:06 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q86DYsVY099164; Thu, 6 Sep 2012 07:35:00 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q86DYfPw043783; Thu, 6 Sep 2012 07:34:41 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: "O. Hartmann" In-Reply-To: <504885AA.1000300@mail.zedat.fu-berlin.de> References: <5047659D.8000107@mail.zedat.fu-berlin.de> <20120906121639.4b7d894d089b2ddeb42f31b4@alkumuna.eu> <504885AA.1000300@mail.zedat.fu-berlin.de> Content-Type: text/plain; charset="us-ascii" Date: Thu, 06 Sep 2012 07:34:41 -0600 Message-ID: <1346938481.59094.117.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: "freebsd >> Current FreeBSD" , freebsd-ports@freebsd.org, Matthieu Volat Subject: Re: Help. Porting "FreeOCL" fails (atomic_ops.h missing, CLANG++ libc++ issues ...) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 13:35:13 -0000 On Thu, 2012-09-06 at 13:14 +0200, O. Hartmann wrote: > I tried to add > > RUN_DEPENDS= > ${LOCALBASE}/lib/libatomic_ops.a:${PORTSDIR}/devel/libatomic_ops > > to my provided Makefile, but this doesn't install the port > devel/libatomic_ops. > This is weird and inconsistent. I follow exact the steps suggested in > the Porter's handbook, the _DEPENDS= section. The above RUN_DEPENDS= > tag > should ensure a check for the existence of the static library > > /usr/local/lib/libatomic_ops.a > > and if not existent, then install it. It doesn't work. Unfortunately, > LIB_DEPENDS is considered for "shared libraries", so it isn't > suitable. > But LIB_DEPENDS get recognized, even if it fails, while RUN_DEPENDS > seems not to be touched by the build process anyway ... I am SO not a ports expert, but I think maybe for a static lib you need BUILD_DEPENDS because it has to be available at build-time rather than run-time. -- Ian From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 13:49:34 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 40A2E106566B for ; Thu, 6 Sep 2012 13:49:34 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from mail.ultra-secure.de (mail.ultra-secure.de [78.47.114.122]) by mx1.freebsd.org (Postfix) with ESMTP id 5C97F8FC18 for ; Thu, 6 Sep 2012 13:49:32 +0000 (UTC) Received: (qmail 29410 invoked by uid 89); 6 Sep 2012 13:49:30 -0000 Received: by simscan 1.4.0 ppid: 29405, pid: 29407, t: 0.0388s scanners: attach: 1.4.0 clamav: 0.97.3/m:54/d:15319 Received: from unknown (HELO suse2.ip-tech.ch) (rainer@ultra-secure.de@212.71.117.1) by mail.ultra-secure.de with ESMTPA; 6 Sep 2012 13:49:30 -0000 Date: Thu, 6 Sep 2012 15:49:30 +0200 From: Rainer Duffner To: Alexander Yerenkow Message-ID: <20120906154930.012e2ad0@suse2.ip-tech.ch> In-Reply-To: References: <20120906123744.3883e075@suse2.ip-tech.ch> X-Mailer: Claws Mail 3.7.10 (GTK+ 2.22.1; x86_64-unknown-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: Where to ask questions about poudriere? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 13:49:34 -0000 Am Thu, 6 Sep 2012 14:59:03 +0300 schrieb Alexander Yerenkow : > I don't see where you specify your zfs tank, is it missed in mail, or > in conf? It slipped into the previous line: ZPOOL=datapool FTPHOST=ftp.ch.freebsd.org FREEBSD_HOST=http://ftp.ch.freebsd.org/ RESOLV_CONF=/etc/resolv.conf BASEFS=/usr/local/poudriere USE_PORTLINT=no USE_TMPFS=yes DISTFILES_CACHE=/usr/ports/distfiles CSUP_HOST=localhost CHECK_CHANGED_OPTIONS=yes PKG_REPO_SIGNING_KEY=/etc/ssl/keys/repo.bla.ch.key CCACHE_DIR=/data/cache/ccache I've somehow missed that poudriere is a shell-script. somewhere down there, it sets PORTSDIR=/ports is that correct? I think it should be /usr/ports... + echo '====>> Starting jail 90amd64' ====>> Starting jail 90amd64 + jrun 0 + [ 1 -ne 1 ] + local network=0 + local ipargs + [ 0 -eq 0 ] + ipargs='ip4.addr=127.0.0.1 ip6.addr=::1' + jail -c persist name=90amd64 ip4.addr=127.0.0.1 ip6.addr=::1 path=/usr/local/poudriere/jails/90amd64 host.hostname=90amd64 allow.sysvipc allow.mount allow.socket_af allow.raw_sockets allow.chflags + [ 1 -eq 1 ] + export STATUS=1 + prepare_jail + export PACKAGE_BUILDING=yes + export USER=root + port_get_base default + [ 1 -ne 1 ] + zfs list -rt filesystem -H -o poudriere:type,poudriere:name,mountpoint datapool/poudriere + awk -v n=default '$1 == "ports" && $2 == n { print $3 }' + PORTSDIR=/ports + POUDRIERED=/usr/local/share/poudriere/../../etc/poudriere.d + [ -z /usr/local/poudriere/jails/90amd64 ] + [ -z /ports ] + [ -z /usr/local/poudriere/data/packages/90amd64-default ] + [ -n '' -a -n yes ] + msg 'Mounting ports filesystems for 90amd64' + echo '====>> Mounting ports filesystems for 90amd64' ====>> Mounting ports filesystems for 90amd64 + do_portbuild_mounts 1 + [ 1 -ne 1 ] + local should_mkdir=1 + [ 1 -eq 1 ] + mkdir -p /ports/packages + mkdir -p /usr/local/poudriere/data/packages/90amd64-default/All + [ -n /usr/ports/distfiles -a -d /usr/ports/distfiles ] + mkdir -p /usr/local/poudriere/jails/90amd64/usr/ports/distfiles + [ -n /data/cache/ccache -a -d /data/cache/ccache ] + mkdir -p /usr/local/poudriere/jails/90amd64/data/cache/ccache + msg 'Mounting ccache from /data/cache/ccache' + echo '====>> Mounting ccache from /data/cache/ccache' ====>> Mounting ccache from /data/cache/ccache + export CCACHE_DIR + mount -t nullfs /ports /usr/local/poudriere/jails/90amd64/usr/ports + mount -t nullfs /usr/local/poudriere/data/packages/90amd64-default /usr/local/poudriere/jails/90amd64/usr/ports/packages + [ -n /usr/ports/distfiles -a -d /usr/ports/distfiles ] + mount -t nullfs /usr/ports/distfiles /usr/local/poudriere/jails/90amd64/usr/ports/distfiles mount: /usr/local/poudriere/jails/90amd64/usr/ports/distfiles: No such file or directory + err 1 'Failed to mount the distfile directory' > 2012/9/6 Rainer Duffner > > > Hi, > > > > I'm trying to get poudriere working with the following settings: > > > > f2d169d8-20d2-41d4-8e43-8a9fc5a2b509# > > cat /usr/local/etc/poudriere.conf |grep -v ^# |grep -v ^$ > > ZPOOL=datapool FTPHOST=ftp.ch.freebsd.org > > FREEBSD_HOST=http://ftp.ch.freebsd.org/ > > RESOLV_CONF=/etc/resolv.conf > > BASEFS=/usr/local/poudriere > > USE_PORTLINT=no > > USE_TMPFS=yes > > DISTFILES_CACHE=/usr/ports/distfiles > > CSUP_HOST=localhost > > CHECK_CHANGED_OPTIONS=yes > > PKG_REPO_SIGNING_KEY=/etc/ssl/keys/repo.bla.ch.key > > CCACHE_DIR=/data/cache/ccache > > > > > > f2d169d8-20d2-41d4-8e43-8a9fc5a2b509# > > cat /usr/local/etc/poudriere.d/make.conf WITH_CCACHE_BUILD=yes > > USE_LOCAL_MK=yes > > > > f2d169d8-20d2-41d4-8e43-8a9fc5a2b509# poudriere bulk -f > > ~/portlist-current-php53-mysql55 -j 90amd64 ====>> Mounting system > > devices for 90amd64 /etc/resolv.conf > > -> /usr/local/poudriere/jails/90amd64/etc/resolv.conf ====>> > > Starting jail 90amd64 ====>> Mounting ports filesystems for 90amd64 > > ====>> Mounting ccache from /data/cache/ccache > > mount: /usr/local/poudriere/jails/90amd64/usr/ports/distfiles: No > > such file or directory ====>> Umounting file systems > > Failed to mount the distfile directory > > You have new mail. > > > > > > f2d169d8-20d2-41d4-8e43-8a9fc5a2b509# zfs list > > NAME USED AVAIL REFER MOUNTPOINT > > datapool 1.51G 47.5G 32K /datapool > > datapool/poudriere 1.51G 47.5G > > 33K /datapool/poudriere datapool/poudriere/data 36K > > 47.5G 36K /usr/local/poudriere/data > > datapool/poudriere/jails 1017M 47.5G > > 31K /datapool/poudriere/jails datapool/poudriere/jails/90amd64 > > 1017M 47.5G 1017M /usr/local/poudriere/jails/90amd64 > > datapool/poudriere/ports 534M 47.5G > > 31K /datapool/poudriere/ports datapool/poudriere/ports/current > > 534M 47.5G 534M /usr/local/poudriere/ports/current > > > > > > What is the problem? > > > > > > > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to > > "freebsd-current-unsubscribe@freebsd.org" > > > > > -- -- Everyware AG Rainer Duffner CISSP, LPI, MCSE Zurlindenstrasse 52a CH-8003 Zurich Telefon +41 44 466 60 25 Telefax +41 44 466 60 10 From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 15:19:20 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C0E4A106566B; Thu, 6 Sep 2012 15:19:20 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) by mx1.freebsd.org (Postfix) with ESMTP id 98CDA8FC17; Thu, 6 Sep 2012 15:19:20 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.5/8.14.5) with ESMTP id q86FJB3R007983; Thu, 6 Sep 2012 08:19:11 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.5/8.14.5/Submit) id q86FJBND007982; Thu, 6 Sep 2012 08:19:11 -0700 (PDT) (envelope-from sgk) Date: Thu, 6 Sep 2012 08:19:11 -0700 From: Steve Kargl To: Roman Divacky Message-ID: <20120906151911.GA7967@troutmask.apl.washington.edu> References: <5046670C.6050500@andric.com> <20120904214344.GA17723@troutmask.apl.washington.edu> <504679CB.90204@andric.com> <20120904221413.GA19395@troutmask.apl.washington.edu> <50471BEE.6030708@andric.com> <20120905221310.GA97847@troutmask.apl.washington.edu> <20120906084312.GA13223@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120906084312.GA13223@freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: Garrett Cooper , Dimitry Andric , freebsd-current@freebsd.org Subject: Re: Compiler performance tests on FreeBSD 10.0-CURRENT X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 15:19:20 -0000 On Thu, Sep 06, 2012 at 10:43:12AM +0200, Roman Divacky wrote: > On Wed, Sep 05, 2012 at 03:13:11PM -0700, Steve Kargl wrote: > > > > Compiling libm on > > > > CPU: AMD Opteron(tm) Processor 248 (2192.01-MHz K8-class CPU) > > Origin = "AuthenticAMD" Id = 0xf5a Family = f Model = 5 Stepping = 10 > > Features=0x78bfbff > MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2> > > AMD Features=0xe0500800 > > > > with default CFLAGS (ie., -O2 -pipe) and -march=opteron. > > Was this compiled as amd64 or i386? Also, can you send me the test case? > So that we can explore the difference. The working theory now is SSE vs FPU > mathematics, but it would be nice to see the testcase. > > Thank you, roman > It was compiled on amd64. I can do the same testing on i386 this weekend. The testcase is not a self-contained piece of code. It uses parts of my floating point test frame. Putting together the testcase may take a few hours. -- Steve From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 15:38:01 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3819C106575B for ; Thu, 6 Sep 2012 15:38:01 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) by mx1.freebsd.org (Postfix) with ESMTP id E0D028FC18 for ; Thu, 6 Sep 2012 15:38:00 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1T9e9W-0000DQ-68 for freebsd-current@freebsd.org; Thu, 06 Sep 2012 17:37:58 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 06 Sep 2012 17:37:58 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 06 Sep 2012 17:37:58 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-current@freebsd.org From: Ivan Voras Date: Thu, 06 Sep 2012 17:37:28 +0200 Lines: 46 Message-ID: References: <20120830141939.GJ64447@ithaqua.etoilebsd.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig8E2AEB4EFF57E59471BCAA43" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:10.0) Gecko/20120213 Thunderbird/10.0 In-Reply-To: <20120830141939.GJ64447@ithaqua.etoilebsd.net> X-Enigmail-Version: 1.3.5 Subject: Re: pkg (aka pkgng) 1.0 released X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 15:38:01 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig8E2AEB4EFF57E59471BCAA43 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 30/08/2012 16:19, Baptiste Daroussin wrote: > Hi all, >=20 > Since Julien Laffaye and I started pkgng lots of things has happened an= d here we > are now. >=20 > After 2 years of development (first commit "Tue Sep 7 2010"), more than= 2000 > commits, 43 different contibutors. The pkgng team is proud to release = pkg-1.0! Hi, It looks like the pkg port installs pkg.conf.sample with the line: PACKAGESITE : http://pkg.freebsd.org/${ABI}/latest =2E.. which is finally a good step in the direction of making pkgng working by default, except that the "pkg.freebsd.org" site doesn't exist in DNS. Instead, pkgbeta.freebsd.org still exists. I suppose one should be a DNS CNAME for the other? --------------enig8E2AEB4EFF57E59471BCAA43 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlBIw0gACgkQ/QjVBj3/HSyzRgCgk0XfbsVPNEy30vr31aSi0YPW ZMoAoIQ0+e79jC4bMKZv1Bx+WNPly2es =V6Nf -----END PGP SIGNATURE----- --------------enig8E2AEB4EFF57E59471BCAA43-- From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 16:44:43 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0170A1065670; Thu, 6 Sep 2012 16:44:43 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) by mx1.freebsd.org (Postfix) with ESMTP id 5E4FD8FC12; Thu, 6 Sep 2012 16:44:42 +0000 (UTC) Received: from seedling.local (host86-186-0-200.range86-186.btcentralplus.com [86.186.0.200]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.5/8.14.5) with ESMTP id q86GibgA035369 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 6 Sep 2012 17:44:37 +0100 (BST) (envelope-from matthew@FreeBSD.org) X-DKIM: OpenDKIM Filter v2.5.2 smtp.infracaninophile.co.uk q86GibgA035369 Authentication-Results: smtp.infracaninophile.co.uk/q86GibgA035369; dkim=none (no signature); dkim-adsp=none X-Authentication-Warning: lucid-nonsense.infracaninophile.co.uk: Host host86-186-0-200.range86-186.btcentralplus.com [86.186.0.200] claimed to be seedling.local Message-ID: <5048D2EC.70109@FreeBSD.org> Date: Thu, 06 Sep 2012 17:44:28 +0100 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 To: Ivan Voras References: <20120830141939.GJ64447@ithaqua.etoilebsd.net> In-Reply-To: X-Enigmail-Version: 1.4.4 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig5F922C8AFD39566E334D9235" X-Virus-Scanned: clamav-milter 0.97.5 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=2.5 required=5.0 tests=AWL,BAYES_00,RCVD_IN_PBL, RCVD_IN_RP_RNBL, RCVD_IN_SORBS_DUL, RDNS_DYNAMIC, SPF_SOFTFAIL autolearn=no version=3.3.2 X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk Cc: freebsd-current@FreeBSD.org Subject: Re: pkg (aka pkgng) 1.0 released X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 16:44:43 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig5F922C8AFD39566E334D9235 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 06/09/2012 16:37, Ivan Voras wrote: > On 30/08/2012 16:19, Baptiste Daroussin wrote: >> Hi all, >> >> Since Julien Laffaye and I started pkgng lots of things has happened a= nd here we >> are now. >> >> After 2 years of development (first commit "Tue Sep 7 2010"), more tha= n 2000 >> commits, 43 different contibutors. The pkgng team is proud to release= pkg-1.0! >=20 > Hi, >=20 > It looks like the pkg port installs pkg.conf.sample with the line: >=20 > PACKAGESITE : http://pkg.freebsd.org/${ABI}/latest >=20 > ... which is finally a good step in the direction of making pkgng > working by default, except that the "pkg.freebsd.org" site doesn't exis= t > in DNS. Instead, pkgbeta.freebsd.org still exists. I suppose one should= > be a DNS CNAME for the other? It's a SRV record: seedling:~:% dig +short IN SRV _http._tcp.pkg.freebsd.org 10 10 80 pkgbeta.FreeBSD.org. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. PGP: http://www.infracaninophile.co.uk/pgpkey --------------enig5F922C8AFD39566E334D9235 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlBI0vQACgkQ8Mjk52CukIxdkACaAnkeSmGvTOThwwKQwpx12DJf z+8AmQGNcYNvfqYThKtdN39lwmzZLZU7 =1D6p -----END PGP SIGNATURE----- --------------enig5F922C8AFD39566E334D9235-- From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 20:23:19 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 64C5D1065674 for ; Thu, 6 Sep 2012 20:23:19 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) by mx1.freebsd.org (Postfix) with ESMTP id 457528FC18 for ; Thu, 6 Sep 2012 20:23:19 +0000 (UTC) Received: from epsilon.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 183311D3DC; Thu, 6 Sep 2012 13:23:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1346962999; bh=ctpPDBGfM8F/lTPfOj4Of61keuZmx1/HbmkpIQ7gj6Y=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=mEFQz8NgMI3IIh9LjY2ocsgZRJw0TXo8OZTy1egBFgr+e2hayluwdh8qtIBi5Ggkx TJ8emf4K4RsWPlQGOqp9ztFsF01bkvlVnyQSdJjaVf5VHMPns9ofOci6VeJSZUIOtl OdYIC6CLeDlewRpzMLTQQnoYsmafZc53qfu9QXzs= Message-ID: <50490636.40803@delphij.net> Date: Thu, 06 Sep 2012 13:23:18 -0700 From: Xin Li Organization: The freeBSD Project User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:10.0.7) Gecko/20120830 Thunderbird/10.0.7 MIME-Version: 1.0 To: Konstantin Belousov References: <20120902103406.GU33100@deviant.kiev.zoral.com.ua> In-Reply-To: <20120902103406.GU33100@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: current@freebsd.org Subject: Re: Bull Mountain (IvyBridge +) random number generator X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 20:23:19 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 09/02/12 03:34, Konstantin Belousov wrote: > It is relatively well known that Ivy Bridge CPUs (Core iX 3XXX) > have built-in hardware random number generator, which is claimed to > be both very fast and high quality. Generator is accessible using > non-privileged RDRAND instruction. It is claimed that CPU performs > sanitization of the random sequence. In particular, it seems that > paranoid AES encryption of the raw random stream, performed by our > padlock driver, is not needed for Bull Mountain (there are hints > that hardware performs it already). > > See > http://spectrum.ieee.org/computing/hardware/behind-intels-new-randomnumber-generator/0 > > http://software.intel.com/en-us/articles/intel-digital-random-number-generator-drng-software-implementation-guide/ > and IA32 ADM. > > Patch at http://people.freebsd.org/~kib/misc/bull_mountain.2.patch > implements support for the generator. I do not own any IvyBridge > machines, so I cannot test. Patch makes both padlock and bull > generators the options, you need to enable IVY_RNG to get support > for the generator. > > I would be interested in seeing reports including verbose boot > dmesg, and some tests of /dev/random quality on the IvyBridge > machines, you can start with > http://lists.gnupg.org/pipermail/gnupg-devel/2000-March/016328.html. CPU: > Intel(R) Core(TM) i7-3610QM CPU @ 2.30GHz (2294.83-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x306a9 Family = 6 Model = 3a Stepping = 9 Features=0xbfebfbff Features2=0x7fbae3bf AMD Features=0x28100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics ... random: [delphij@epsilon] ~> dd if=/dev/random bs=1m count=256 | ./ent 256+0 records in 256+0 records out 268435456 bytes transferred in 8.330823 secs (32221961 bytes/sec) Entropy = 7.999999 bits per byte. Optimum compression would reduce the size of this 268435456 byte file by 0 percent. Chi square distribution for 268435456 samples is 237.19, and randomly would exceed this value 78.17 percent of the times. Arithmetic mean value of data bytes is 127.4968 (127.5 = random). Monte Carlo value for Pi is 3.141569721 (error 0.00 percent). Serial correlation coefficient is -0.000080 (totally uncorrelated = 0.0). [delphij@epsilon] ~> dd if=/dev/random bs=1m count=256 | ./ent 256+0 records in 256+0 records out 268435456 bytes transferred in 8.110786 secs (33096109 bytes/sec) Entropy = 7.999999 bits per byte. Optimum compression would reduce the size of this 268435456 byte file by 0 percent. Chi square distribution for 268435456 samples is 265.06, and randomly would exceed this value 31.95 percent of the times. Arithmetic mean value of data bytes is 127.4982 (127.5 = random). Monte Carlo value for Pi is 3.141918140 (error 0.01 percent). Serial correlation coefficient is 0.000005 (totally uncorrelated = 0.0). [delphij@epsilon] ~> dd if=/dev/random bs=1m count=256 | ./ent 256+0 records in 256+0 records out 268435456 bytes transferred in 8.094252 secs (33163714 bytes/sec) Entropy = 7.999999 bits per byte. Optimum compression would reduce the size of this 268435456 byte file by 0 percent. Chi square distribution for 268435456 samples is 263.17, and randomly would exceed this value 34.92 percent of the times. Arithmetic mean value of data bytes is 127.4969 (127.5 = random). Monte Carlo value for Pi is 3.141545045 (error 0.00 percent). Serial correlation coefficient is 0.000017 (totally uncorrelated = 0.0). - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBCAAGBQJQSQY2AAoJEG80Jeu8UPuzHTUH/37b3iinQ3/yjc2tfTjKAMZh KJGEzZ1hlr8Ifoax3ul27U7Mpyss85Vza+tICeiyDpPulFlKuJa9lFfadNXIiDqR AAB4PtK+cZ8uyVze00sstU+7tK7AqKCyuz/yL6fzK2h2Bx8mYVgE3UTK+DOwQcEa 4Y0pFlO7gPnw1NGK6T7Ofnl/s9wum3JWELPhaTmo5L11JioXnufTmsJpB2MzqSxT iK0B0FCzF32e1Hl5HNNEMbfx7Rrx+Pf1OzdhP+/1+WHdXn8qtr8htsmsA/4zV+pT jAHHGuPxNaFmb2xyEZtQerPPdexoadWjrNlFQtl2gsVyMrWYBX2PyT3n3bbos50= =eiAK -----END PGP SIGNATURE----- From owner-freebsd-current@FreeBSD.ORG Thu Sep 6 20:59:51 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1D0951065741; Thu, 6 Sep 2012 20:59:51 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id C6DFB8FC08; Thu, 6 Sep 2012 20:59:50 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.69) with esmtp (envelope-from ) id <1T9jAz-0001H7-F2>; Thu, 06 Sep 2012 22:59:49 +0200 Received: from e178003010.adsl.alicedsl.de ([85.178.3.10] helo=thor.walstatt.dyndns.org) by inpost2.zedat.fu-berlin.de (Exim 4.69) with esmtpsa (envelope-from ) id <1T9jAz-0002AS-9p>; Thu, 06 Sep 2012 22:59:49 +0200 Message-ID: <50490EBF.6090709@zedat.fu-berlin.de> Date: Thu, 06 Sep 2012 22:59:43 +0200 From: "O. Hartmann" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120810 Thunderbird/14.0 MIME-Version: 1.0 To: Ports FreeBSD , "free >> Current FreeBSD" X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig7AA9EFCAD3D96893A4F9CA11" X-Originating-IP: 85.178.3.10 Cc: Subject: fetch(3): Authentication error when URL https:// X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2012 20:59:51 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig7AA9EFCAD3D96893A4F9CA11 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello. Creating a port, I need to fectch sources from a site whos URL is https://xxx.xxx.xxx. Doing so, I end up with an "Authentication error". This makes the fetch process in the port's Makefile impossible. I tried to fetch the source tar-ball via "wget(1)", but this also fails, wget suggests to use option --no-check-certificate: Connecting to launchpad.net (launchpad.net)|91.189.89.223|:443... connect= ed. ERROR: cannot verify launchpad.net's certificate, issued by `/C=3DUS/ST=3DArizona/L=3DScottsdale/O=3DGoDaddy.com, Inc./OU=3Dhttp://certificates.godaddy.com/repository/CN=3DGo Daddy Secure= Certification Authority/serialNumber=3D07969287': Self-signed certificate encountered. To connect to launchpad.net insecurely, use `--no-check-certificate'. Doing so succeeds. Is there a possibility to enable this --no-check-certificate" with fetch, too? Regards, Oliver --------------enig7AA9EFCAD3D96893A4F9CA11 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBAgAGBQJQSQ7EAAoJEOgBcD7A/5N8N9AIAKW/BXW0wD4gd2Db3EXalR6M BgFP/jm3T2W3rNgYq7NKaXB3PU/PgoaxNX5RunmNsMyoboUcE+YJ2l3Ha2C2n4ri sIA41niAdL42ps5RWNj4lsOgRwZARcmH7Eh8x0W8DxpfBzN9Sj57dCQExcNBQ3iI gqvfaGPhF1mXw6T8PXy7NRWWX00XUBQS7wvT1XUYHq9kC3gelHd/+EMuhQrATXjk dm1Sh3fZBEjkMhp76eFWxON7qaMwofn/U0E6gBB66/xGm2CONXPr/p5UULXMUBer 3PZT50EnkSizWkiPNTWqvvCeW7whclThiUqQqoFZVSt+WttPvGddlyeojdUyTcA= =CRER -----END PGP SIGNATURE----- --------------enig7AA9EFCAD3D96893A4F9CA11-- From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 08:35:26 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AC0FB1065670 for ; Fri, 7 Sep 2012 08:35:26 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id 6069D8FC0C for ; Fri, 7 Sep 2012 08:35:26 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.69) for freebsd-current@FreeBSD.org with esmtp (envelope-from ) id <1T9u27-0008Or-PD>; Fri, 07 Sep 2012 10:35:23 +0200 Received: from e178006154.adsl.alicedsl.de ([85.178.6.154] helo=thor.walstatt.dyndns.org) by inpost2.zedat.fu-berlin.de (Exim 4.69) for freebsd-current@FreeBSD.org with esmtpsa (envelope-from ) id <1T9u27-0001zD-K7>; Fri, 07 Sep 2012 10:35:23 +0200 Message-ID: <5049B1C2.6030604@zedat.fu-berlin.de> Date: Fri, 07 Sep 2012 10:35:14 +0200 From: "O. Hartmann" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120810 Thunderbird/14.0 MIME-Version: 1.0 To: Current FreeBSD X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig6DB24A3102F5A069563181BA" X-Originating-IP: 85.178.6.154 Cc: Subject: make: don't know how to make add-plist-buildinfo. Stop X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 08:35:26 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig6DB24A3102F5A069563181BA Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On all portupgrades, port installations I receive this error since today on FreeBSD 10.0-CURRENT amd64 r240150M: make: don't know how to make add-plist-buildinfo. Stop *** [reinstall] Error code 2 What is missing? Oliver --------------enig6DB24A3102F5A069563181BA Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBAgAGBQJQSbHLAAoJEOgBcD7A/5N8YfUH/igIX9AExyCyC7qeU09usu6N YGOI6LREoOw7CJWt3qng8equcP6vX7wsnFHhb/aAP5N2NhY9S766ugBcYHWFVjEU DBuAhWhBViheUSdiARER8OLnFAwQqt77PL3GPqmyh1Gb8uME5Uu+CCtjim5j38qC r003SJbqb8FUQryX7JCY2JepN7cICjaWoqw3OV3UZlOqxSBmdszkhAajLfci6mzx Lx85isiprGtlMfX/nBclspePC1DPwfvuV/mvjlAV1a3GDoGjXi3ztS90KiLdVALm +nQo2aX+CE3gwg8XcSN3pxUBPLk6b0zz3G9FVGilHcAbBdCjeJFtBEQU1i5jgSU= =/bZv -----END PGP SIGNATURE----- --------------enig6DB24A3102F5A069563181BA-- From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 04:34:15 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BDE22106564A; Fri, 7 Sep 2012 04:34:15 +0000 (UTC) (envelope-from duchateau.olivier@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 8CC3C8FC12; Fri, 7 Sep 2012 04:34:15 +0000 (UTC) Received: by dadr6 with SMTP id r6so1587399dad.13 for ; Thu, 06 Sep 2012 21:34:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=wu+QQhJpvmu8ix6eKfRKwfT7SPzbkwArVBB84vO6ya0=; b=L7/g46BvngkzYXiivwZdJFqEZZrrNzqBfDy8HJb+q05/ohDbu3yw8YLGW3LHF2GfOp tS2nKTTCwmVOgZAb5sSzd8aMRSosV1fq6XOsRS0uwaFfaKLfs/+YfHAPw9jQdGGhhQsV YfS0gNBpvFYB529hH6C3qHBnXuoBhbL9AP+Tam1IbVCv1ROAEJszIPOYf8U8PmqAASs3 4hWaxqzJOnPU/lyTKFuvWwAWTNQ0hPzw9VqBGPDbf8a3P806z+zpyBALtBb3weMuwcNb Tk9udTljUHFUiW7ea5OFJhKEYtDz4wEtcmSXaSQpv4yXcs3S+pDnXMqc4qi6QsVomJpl xKBw== MIME-Version: 1.0 Received: by 10.68.226.167 with SMTP id rt7mr8161379pbc.146.1346992454743; Thu, 06 Sep 2012 21:34:14 -0700 (PDT) Received: by 10.66.220.103 with HTTP; Thu, 6 Sep 2012 21:34:14 -0700 (PDT) In-Reply-To: <50490EBF.6090709@zedat.fu-berlin.de> References: <50490EBF.6090709@zedat.fu-berlin.de> Date: Fri, 7 Sep 2012 06:34:14 +0200 Message-ID: From: Olivier Duchateau To: "O. Hartmann" Content-Type: text/plain; charset=ISO-8859-1 X-Mailman-Approved-At: Fri, 07 Sep 2012 11:04:21 +0000 Cc: "free >> Current FreeBSD" , Ports FreeBSD Subject: Re: fetch(3): Authentication error when URL https:// X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 04:34:15 -0000 2012/9/6 O. Hartmann : > Hello. > > Creating a port, I need to fectch sources from a site whos URL is > https://xxx.xxx.xxx. > > Doing so, I end up with an "Authentication error". This makes the fetch > process in the port's Makefile impossible. > > I tried to fetch the source tar-ball via "wget(1)", but this also fails, > wget suggests to use option --no-check-certificate: > > Connecting to launchpad.net (launchpad.net)|91.189.89.223|:443... connected. > ERROR: cannot verify launchpad.net's certificate, issued by > `/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, > Inc./OU=http://certificates.godaddy.com/repository/CN=Go Daddy Secure > Certification Authority/serialNumber=07969287': > Self-signed certificate encountered. > To connect to launchpad.net insecurely, use `--no-check-certificate'. Add in Makefile FETCH_ARGS= -p > > > Doing so succeeds. Is there a possibility to enable this > --no-check-certificate" with fetch, too? > > Regards, > Oliver > -- olivier From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 09:41:25 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76AD11065674 for ; Fri, 7 Sep 2012 09:41:25 +0000 (UTC) (envelope-from ohartman@mail.zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id 1F0A98FC12 for ; Fri, 7 Sep 2012 09:41:24 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.69) for freebsd-current@freebsd.org with esmtp (envelope-from ) id <1T9v3z-0006Kh-VE>; Fri, 07 Sep 2012 11:41:24 +0200 Received: from telesto.geoinf.fu-berlin.de ([130.133.86.198]) by inpost2.zedat.fu-berlin.de (Exim 4.69) for freebsd-current@freebsd.org with esmtpsa (envelope-from ) id <1T9v3z-0006hv-Qs>; Fri, 07 Sep 2012 11:41:23 +0200 Message-ID: <5049C13E.5010808@mail.zedat.fu-berlin.de> Date: Fri, 07 Sep 2012 11:41:18 +0200 From: "O. Hartmann" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120810 Thunderbird/14.0 MIME-Version: 1.0 To: Current FreeBSD X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig19310305CB1D4343BC0E2C29" X-Originating-IP: 130.133.86.198 X-Mailman-Approved-At: Fri, 07 Sep 2012 11:14:55 +0000 Subject: FreeBSD 10.0-CURRENT: CLANG and port/clang weirdness! X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 09:41:25 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig19310305CB1D4343BC0E2C29 Content-Type: multipart/mixed; boundary="------------070608030709050508060704" This is a multi-part message in MIME format. --------------070608030709050508060704 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Building ports not explicitely enabling USE_GCC=3D4.6+ are considered using the system's LLVM/CLANG, which is clang 3.2 in our installation (FreeBSD 10.0-CURRENT #0 r240164), but since some ports require the special ports devel/llvm and lang/clang, LLVM 3.1 and clang 3.1 get installed and 3.1 is used instead the system's 3.2 whenever "clang", "clang++" is invoked. Following the WIKI at http://wiki.freebsd.org/BuildingFreeBSDWithClang introduces the usage of CC=3Dclang instead of CC=3D/usr/bin/clang CXX=3Dclang++ instead of CXX=3D/usr/bin/clang++ CPP=3Dclang-ccp instead of CPP=3D/usr/bin/clang-ccp Is this intended? Since I can not simply change the search patch - I need to have /usr/local/bin before /usr/bin, is there a way to avoid this confusion? Building software with makefiles or self-created ports always refer to the port's LLVM/CLANG, which is LLVM/CLANG 3.1 due to some reuqirements of several ports. I'm really confused. Am I missing some special knob here and fell into this pit by not-having-the-knowledge? Or is it really this messy? Well, I'd like to stay with the core's LLVM/CLANG, which is 3.2 whenever I simply issue "clang" or "clang++" (a pity that LLVM isn't completely installed). My /etc/make.conf portion looks this: ## ## CLANG ## =2Eif !defined(NO_CLANG) =2Eif !defined(CC) || ${CC} =3D=3D "cc" CC=3D /usr/bin/clang =2Eendif =2Eif !defined(CXX) || ${CXX} =3D=3D "c++" CXX=3D /usr/bin/clang++ =2Eendif =2Eif !defined(CPP) || ${CPP} =3D=3D "cpp" CPP=3D /usr/bin/clang-cpp =2Eendif ## Don't die on warnings #NO_WERROR=3D #WERROR=3D ## Don't forget this when using Jails! #NO_FSCHG=3D CFLAGS=3D -O3 -pipe # -fno-strict-aliasing COPTFLAGS=3D -O3 -pipe # #CXXFLAGS+=3D -stdlib=3Dlibc++ =2Eendif My /etc/src.conf is attached. If there is a clean way to distinguish, please help me. Regards, Oliver --------------070608030709050508060704 Content-Type: text/plain; charset=us-ascii; name="src.conf" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="src.conf" #.if !defined(NO_CLANG) #CC=3D /usr/bin/clang #CXX=3D /usr/bin/clang++ #CPP=3D /usr/bin/clang-cpp ## #CFLAGS+=3D -O3 -pipe # -fno-strict-aliasing #COPTFLAGS+=3D -O3 -pipe #CXXFLAGS+=3D -stdlib=3Dlibc++ # ## Don't die on warnings #NO_WERROR=3D #WERROR=3D # ## Don't forget this when using Jails! #NO_FSCHG=3D # need clang++ to be built #WITH_LIBCPLUSPLUS=3D YES # #WITH_CLANG_IS_CC=3D YES #.endif # WITH_CLANG=3D YES WITH_CLANG_EXTRAS=3D YES # WITH_LIBCPLUSPLUS=3D YES # WITH_BIND_LARGE_FILE=3D YES WITH_BIND_SIGCHASE=3D YES WITH_BIND_LIBS=3D YES # WITH_IDEA=3D YES WITH_HESIOD=3D YES # #WITH_ICONV=3D YES #WITH_BSD_GREP=3D YES WITH_BSD_SORT=3D YES # WITH_BSDCONFIG=3D YES # #WITH_OFED=3D YES # MALLOC_PRODUCTION=3D YES # PORTS_MODULES=3D emulators/virtualbox-ose-kmod x11/nvidia-driver --------------070608030709050508060704-- --------------enig19310305CB1D4343BC0E2C29 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBAgAGBQJQScFDAAoJEOgBcD7A/5N81nAIAMUBZ7UROq96AZuDkB9NxPuA nwnGTm2lfV2sqSN0C2VXZVayzvgZ9ixWDyzq6huFYIdV5hd7M6I+9PSDK8piIWpA o2//FKilNbiqE/VHPqvyd02/52Jm3bMB/D+2Y4/kGigjLcsKbMnpXG6Wm7dcc59X +UEf/N0AYa7it+C0AYk7lQVG8nsjU7it3StcFLAQafQM9P1ePuIyzqCrNh8kIJ/d /wNmwTdYCWT+GWS6hYbr0/I9CHT8+FjPFLpHuJSy3f2JEH4JmmOuKpiFLQx+Ea8r 6De3DQD48yuRE73fE3kpkBcT9Jt+4uO3axW0o5GnHEnmyoT5so0NTeCwTQEQQEI= =mPUB -----END PGP SIGNATURE----- --------------enig19310305CB1D4343BC0E2C29-- From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 11:30:58 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 93734106566C for ; Fri, 7 Sep 2012 11:30:58 +0000 (UTC) (envelope-from freebsd-current@m.gmane.org) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) by mx1.freebsd.org (Postfix) with ESMTP id 473248FC0A for ; Fri, 7 Sep 2012 11:30:58 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1T9wm2-0005cV-Hy for freebsd-current@freebsd.org; Fri, 07 Sep 2012 13:30:58 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 07 Sep 2012 13:30:58 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 07 Sep 2012 13:30:58 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-current@freebsd.org From: Ivan Voras Date: Fri, 07 Sep 2012 13:30:39 +0200 Lines: 46 Message-ID: References: <20120830141939.GJ64447@ithaqua.etoilebsd.net> <5048D2EC.70109@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigFE20672725997150091D3EF7" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120812 Thunderbird/14.0 In-Reply-To: <5048D2EC.70109@FreeBSD.org> X-Enigmail-Version: 1.4.3 Subject: Re: pkg (aka pkgng) 1.0 released X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 11:30:58 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigFE20672725997150091D3EF7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 06/09/2012 18:44, Matthew Seaman wrote: > On 06/09/2012 16:37, Ivan Voras wrote: >> Hi, >> >> It looks like the pkg port installs pkg.conf.sample with the line: >> >> PACKAGESITE : http://pkg.freebsd.org/${ABI}/latest >> >> ... which is finally a good step in the direction of making pkgng >> working by default, except that the "pkg.freebsd.org" site doesn't exi= st >> in DNS. Instead, pkgbeta.freebsd.org still exists. I suppose one shoul= d >> be a DNS CNAME for the other? >=20 > It's a SRV record: >=20 > seedling:~:% dig +short IN SRV _http._tcp.pkg.freebsd.org > 10 10 80 pkgbeta.FreeBSD.org. Hi, What are the benefits of doing it this way? --------------enigFE20672725997150091D3EF7 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlBJ2t8ACgkQ/QjVBj3/HSxGngCfTuCyTBukY4aAwSSgPvxGYvMS Z/cAniXuh14MAGwqSvEnD27Ab+qbOfqO =v0nA -----END PGP SIGNATURE----- --------------enigFE20672725997150091D3EF7-- From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 11:54:25 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 868F2106564A; Fri, 7 Sep 2012 11:54:25 +0000 (UTC) (envelope-from m.seaman@infracaninophile.co.uk) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) by mx1.freebsd.org (Postfix) with ESMTP id E38A98FC0A; Fri, 7 Sep 2012 11:54:24 +0000 (UTC) Received: from rufus.webfusion.com (mail.heartinternet.co.uk [79.170.40.31]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.5/8.14.5) with ESMTP id q87Bs9OV057121 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 7 Sep 2012 12:54:16 +0100 (BST) (envelope-from m.seaman@infracaninophile.co.uk) X-DKIM: OpenDKIM Filter v2.5.2 smtp.infracaninophile.co.uk q87Bs9OV057121 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=infracaninophile.co.uk; s=201001-infracaninophile; t=1347018856; bh=UmTZsRl3dexE7BuduZhKanH8hV9300mMX5YWI4F6+jQ=; h=Date:From:To:CC:Subject:References:In-Reply-To:Content-Type: Message-ID:Mime-Version; b=ucsm9IVdO8ZWuPye6EhUUfXHQx0eMWcsVauDI/DxHh7ds5gu7Ol38NSIbJIFpue/t JOLjLFRB/wB+i1H5HBZC12TMFRniVOm1/ImFuKdfzshE3fEnOXM+Mr566AZlBrR/zz Dve4hjUn8wADpBygzkQ8er6rczCjiz13FMaVHExI= X-Authentication-Warning: lucid-nonsense.infracaninophile.co.uk: Host mail.heartinternet.co.uk [79.170.40.31] claimed to be rufus.webfusion.com Message-ID: <5049E060.9020602@infracaninophile.co.uk> Date: Fri, 07 Sep 2012 12:54:08 +0100 From: Matthew Seaman User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120831 Thunderbird/15.0 MIME-Version: 1.0 To: Ivan Voras References: <20120830141939.GJ64447@ithaqua.etoilebsd.net> <5048D2EC.70109@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97.5 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00,DKIM_ADSP_ALL, DKIM_SIGNED,SPF_FAIL,T_DKIM_INVALID autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk Cc: freebsd-current@freebsd.org Subject: Re: pkg (aka pkgng) 1.0 released X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 11:54:25 -0000 On 09/07/12 12:30, Ivan Voras wrote: > On 06/09/2012 18:44, Matthew Seaman wrote: >> On 06/09/2012 16:37, Ivan Voras wrote: > >>> Hi, >>> >>> It looks like the pkg port installs pkg.conf.sample with the line: >>> >>> PACKAGESITE : http://pkg.freebsd.org/${ABI}/latest >>> >>> ... which is finally a good step in the direction of making pkgng >>> working by default, except that the "pkg.freebsd.org" site doesn't exist >>> in DNS. Instead, pkgbeta.freebsd.org still exists. I suppose one should >>> be a DNS CNAME for the other? >> >> It's a SRV record: >> >> seedling:~:% dig +short IN SRV _http._tcp.pkg.freebsd.org >> 10 10 80 pkgbeta.FreeBSD.org. > > Hi, > > What are the benefits of doing it this way? Yeah -- it's a bit OTT right now given there's just the one publicly available pkg repository available. It will pay off later when there are more pkg repositories available -- repositories can be added to (or removed from) the list in the SRV record without end-users having to know the details. It may also be possible to replicate what portupdate has done and use geolocation based services to steer end users to a nearby repository site automatically. Cheers, Matthew From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 12:02:24 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 80A361065670; Fri, 7 Sep 2012 12:02:24 +0000 (UTC) (envelope-from m.seaman@infracaninophile.co.uk) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) by mx1.freebsd.org (Postfix) with ESMTP id 04B928FC0C; Fri, 7 Sep 2012 12:02:23 +0000 (UTC) Received: from rufus.webfusion.com (mail.heartinternet.co.uk [79.170.40.31]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.5/8.14.5) with ESMTP id q87C2KAI057340 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 7 Sep 2012 13:02:20 +0100 (BST) (envelope-from m.seaman@infracaninophile.co.uk) X-DKIM: OpenDKIM Filter v2.5.2 smtp.infracaninophile.co.uk q87C2KAI057340 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=infracaninophile.co.uk; s=201001-infracaninophile; t=1347019340; bh=T7p1xl2UNDGUSm9M7UQi07FCy+aifjKRhkCP1J8FjJI=; h=Date:From:To:CC:Subject:References:In-Reply-To:Content-Type: Message-ID:Mime-Version; b=aMv57E1fpKpxzO+PT52XsdPbZkf2Z/Ka97K3ADHLLtw2ysfdSi7Kum47e9rnWiFi4 Utj7vHaK7h1zPqyXoy8y4uUYq7ZRFWqF+ZlN0Fwb03wgH4UXTjdhnw7T9QW0jrCAQW u84pIF5L4NJqoy2W/FDiEKdn6fZxB0xRkIGAcZoI= X-Authentication-Warning: lucid-nonsense.infracaninophile.co.uk: Host mail.heartinternet.co.uk [79.170.40.31] claimed to be rufus.webfusion.com Message-ID: <5049E24C.2020709@infracaninophile.co.uk> Date: Fri, 07 Sep 2012 13:02:20 +0100 From: Matthew Seaman User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120831 Thunderbird/15.0 MIME-Version: 1.0 To: Ivan Voras References: <20120830141939.GJ64447@ithaqua.etoilebsd.net> <5048D2EC.70109@FreeBSD.org> <5049E060.9020602@infracaninophile.co.uk> In-Reply-To: <5049E060.9020602@infracaninophile.co.uk> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.97.5 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-0.9 required=5.0 tests=AWL,BAYES_00,DKIM_ADSP_ALL, DKIM_SIGNED,SPF_FAIL,T_DKIM_INVALID autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk Cc: freebsd-current@freebsd.org Subject: Re: pkg (aka pkgng) 1.0 released X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 12:02:24 -0000 On 09/07/12 12:54, Matthew Seaman wrote: > It may also be possible to replicate what portupdate has done and use > geolocation based services to steer end users to a nearby repository > site automatically. D'Oh! I meant portsnap of course. Matthew From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 12:03:10 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F2918106564A for ; Fri, 7 Sep 2012 12:03:09 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9E0068FC2B for ; Fri, 7 Sep 2012 12:03:09 +0000 (UTC) Received: by vbmv11 with SMTP id v11so4265803vbm.13 for ; Fri, 07 Sep 2012 05:03:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; bh=LQ6AoeoH+wZbxdpW8jnJSpYzzs9HfacXrnLFooXky+8=; b=AEeJ6iqljNV90loN8nCCIoDmX3LKKMZ/aXZUWc5ZPRzjVsmu5PhihMIXGlvQjDjOub 2ocWFpWXvTrAwNKNf+zRGbhlIiClVaBoI3ZJU9FCJQ8Vm5aacXWW33BROZn7P7DtZcYU oJewBlbGm82bAAV1fnEAM8bbTTSnHUzHvco3NG0bqCwO4JE1R28dGnt9zgQsuKrPpMIQ 6cAiFqLpCCn0219pb1tF/lBHkynTGabfM65xwIE+UTt6vpzgKS/VQ/GcNP3y9HinPSYH 1WcHDDQ9hn6z7wUMvTvNj5MyAVF0gnhZobMZAzuMHs3C0VBYulUmYdVAFzzfaADyE5Z8 T3Lg== Received: by 10.221.10.13 with SMTP id oy13mr6919217vcb.14.1347019388881; Fri, 07 Sep 2012 05:03:08 -0700 (PDT) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.59.0.37 with HTTP; Fri, 7 Sep 2012 05:02:28 -0700 (PDT) In-Reply-To: <5049E060.9020602@infracaninophile.co.uk> References: <20120830141939.GJ64447@ithaqua.etoilebsd.net> <5048D2EC.70109@FreeBSD.org> <5049E060.9020602@infracaninophile.co.uk> From: Ivan Voras Date: Fri, 7 Sep 2012 14:02:28 +0200 X-Google-Sender-Auth: P0uIOEdPEHGAXl4MazSs6cuGWdo Message-ID: To: Matthew Seaman Content-Type: text/plain; charset=UTF-8 Cc: freebsd-current@freebsd.org Subject: Re: pkg (aka pkgng) 1.0 released X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 12:03:10 -0000 On 7 September 2012 13:54, Matthew Seaman wrote: > On 09/07/12 12:30, Ivan Voras wrote: >> On 06/09/2012 18:44, Matthew Seaman wrote: >>> On 06/09/2012 16:37, Ivan Voras wrote: >> >>>> Hi, >>>> >>>> It looks like the pkg port installs pkg.conf.sample with the line: >>>> >>>> PACKAGESITE : http://pkg.freebsd.org/${ABI}/latest >>>> >>>> ... which is finally a good step in the direction of making pkgng >>>> working by default, except that the "pkg.freebsd.org" site doesn't exist >>>> in DNS. Instead, pkgbeta.freebsd.org still exists. I suppose one should >>>> be a DNS CNAME for the other? >>> >>> It's a SRV record: >>> >>> seedling:~:% dig +short IN SRV _http._tcp.pkg.freebsd.org >>> 10 10 80 pkgbeta.FreeBSD.org. >> >> Hi, >> >> What are the benefits of doing it this way? > > Yeah -- it's a bit OTT right now given there's just the one publicly > available pkg repository available. It will pay off later when there > are more pkg repositories available -- repositories can be added to (or > removed from) the list in the SRV record without end-users having to > know the details. > > It may also be possible to replicate what portupdate has done and use > geolocation based services to steer end users to a nearby repository > site automatically. Ok, but all that can be done the "normal" way with A records. As far as I can tell, the intended benefits of the SRV record system is to disentangle services and hosts, so that, e.g. the same user-visible DNS name (e.g. "pkg.freebsd.org") resolves to a different host if asked for the HTTP service and the FTP service. I suppose this could work in FreeBSD's case if the record was created differently, instad of the "normal" _http._tcp record, introduce a new one, e.g. _pkgng._tcp, so when a web browser visits "pkg.freebsd.org" it gets a regular web page or some other user-visible content, and the specially made pkg client will resolve it to another service and another (possibly) server. This also can be done without the SRV record, by e.g. inspecting client's HTTP headers. I'm not saying that the SRV record is technically wrong in this case, I just don't see how is it useful (and surely there will be others trying to ping "pkg.freebsd.org" and complaining when it fails to resolve). From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 12:29:39 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F02D3106564A for ; Fri, 7 Sep 2012 12:29:38 +0000 (UTC) (envelope-from yerenkow@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id B163C8FC0A for ; Fri, 7 Sep 2012 12:29:38 +0000 (UTC) Received: by iayy25 with SMTP id y25so3946296iay.13 for ; Fri, 07 Sep 2012 05:29:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=xSXDpFKAfxpJmpk2l5XcWmnufJt6yT4Kfh8/4eeI6Ys=; b=pwgYvJ8U7nDPy1jRoh50VcDbJGq5/hUD5g4qfKvzy1xJ6rRzfVDmr01biwrHG7yY7P PlQGYawZhhV5Tk2gePD59rhJ/fdIu0MaDSZgG7AcolROxDcP51jLcZej2E3JWqfNWYC5 onC7NwIMobacmvpUbvkeivvHRWJaZubFCM2kJ1H1L4j0MmUfEqGhcwzM/j5k9cx2k6ob Idm5QR/O5chM4Fo/Lq0bYrwUFlkm5Pq012NwByd2I4cU8VIyE/lim0K/Fa9Bi+sbbWd/ njqL47TM54qcPV4PaFiuQogCja6RkPtc1LGfOgYCPaGpRWOs5ZMnUwz/tbLLgA2ATzeR Kh8g== MIME-Version: 1.0 Received: by 10.50.219.230 with SMTP id pr6mr8040187igc.29.1347020977791; Fri, 07 Sep 2012 05:29:37 -0700 (PDT) Received: by 10.64.28.36 with HTTP; Fri, 7 Sep 2012 05:29:37 -0700 (PDT) In-Reply-To: References: <20120830141939.GJ64447@ithaqua.etoilebsd.net> <5048D2EC.70109@FreeBSD.org> <5049E060.9020602@infracaninophile.co.uk> Date: Fri, 7 Sep 2012 15:29:37 +0300 Message-ID: From: Alexander Yerenkow To: freebsd-current@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: pkg (aka pkgng) 1.0 released X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 12:29:39 -0000 I just found out, that first call of pkg (which also known as bootstrap) became interactive, and without options. This kinda brokes a bit my automated image creation scripts (which builds images from svn). Would it be nice to have non-interactive bootstrap, maybe pkg --bootstrap or something similar. About interactivity - yes, currently it's one question, yes/no, but I think it's better to make both way now, from beginning rather than later. Thanks. -- Regards, Alexander Yerenkow From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 13:08:23 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D1696106566C for ; Fri, 7 Sep 2012 13:08:23 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9D44A8FC22 for ; Fri, 7 Sep 2012 13:08:23 +0000 (UTC) Received: by pbbrp2 with SMTP id rp2so4230043pbb.13 for ; Fri, 07 Sep 2012 06:08:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=hPEvjgUPJsq4jhH4kIZ1U+ZRNGifqfj8aN3CrB0XTKg=; b=dF1KCKH7CtuFVz3WPlPClp+ZWdApEi27vhRZx/VGEV9haCo23gEzg2UNTiJK0ZMnoN +2eWwJ8qQrI1oGdtFu/yGIWE0G9x6XGOCqbH6oBMWX152w2V4b1ojXae27RT7Lq9EvOb wXljT7pwKsAvzPdPiDfG+FNj+dIq5bzoNVASc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-gm-message-state; bh=hPEvjgUPJsq4jhH4kIZ1U+ZRNGifqfj8aN3CrB0XTKg=; b=i6guBIYlMm+fPor/NvCzxmjiwevMAu7yR92hmeepyNwtL0B3ZNq7urq+pDwyFexAo2 4FsQj/bnKSPAp55/t8sMAcZrvuiXkUrQBtzSQ0QEZrFJiW6EKjSpe9UbYc7dNI3fX+R7 8SQWgrnLfBWoX+dnDxXQHUvZoss5H/IxZPXJ1oRDRNCePtmECuRY3aZ2V1pazoPssSQm v4w3Dps51w8tYSq0BHJSZADxTfU4KzCeo5tWHWNtIVnKRhDb1t0tA2JdIjLYSvskV/bI o2RrPwAe7oJVcPTN9NKTm+Uw3c5LVxZJW6WPCGcW43c6GebkNzpX9bpB+lcsXEEhs6rI XuxQ== Received: by 10.68.242.231 with SMTP id wt7mr9803428pbc.99.1347023302931; Fri, 07 Sep 2012 06:08:22 -0700 (PDT) MIME-Version: 1.0 Received: by 10.67.4.227 with HTTP; Fri, 7 Sep 2012 06:07:52 -0700 (PDT) In-Reply-To: <5049B1C2.6030604@zedat.fu-berlin.de> References: <5049B1C2.6030604@zedat.fu-berlin.de> From: Eitan Adler Date: Fri, 7 Sep 2012 09:07:52 -0400 Message-ID: To: "O. Hartmann" Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQkucUgMgaRXGVg+T11qQIviY4DQLa6LPmSowpgPqWR8mnkTvZplKbSGlBMbbiStp7tuWwym Cc: Current FreeBSD Subject: Re: make: don't know how to make add-plist-buildinfo. Stop X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 13:08:23 -0000 On 7 September 2012 04:35, O. Hartmann wrote: > On all portupgrades, port installations I receive this error since today > on FreeBSD 10.0-CURRENT amd64 r240150M: > > make: don't know how to make add-plist-buildinfo. Stop > *** [reinstall] Error code 2 Fixed already. update your ports tree. -- Eitan Adler From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 14:22:03 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E15A106566B for ; Fri, 7 Sep 2012 14:22:03 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-7.mit.edu (DMZ-MAILSEC-SCANNER-7.MIT.EDU [18.7.68.36]) by mx1.freebsd.org (Postfix) with ESMTP id B8FBF8FC0A for ; Fri, 7 Sep 2012 14:22:02 +0000 (UTC) X-AuditID: 12074424-b7f856d000000942-ce-504a0304b11e Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) by dmz-mailsec-scanner-7.mit.edu (Symantec Messaging Gateway) with SMTP id FC.73.02370.4030A405; Fri, 7 Sep 2012 10:21:56 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id q87ELttY012875; Fri, 7 Sep 2012 10:21:55 -0400 Received: from multics.mit.edu (MULTICS.MIT.EDU [18.187.1.73]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id q87ELrQK026933 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 7 Sep 2012 10:21:54 -0400 (EDT) Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id q87ELqnd026864; Fri, 7 Sep 2012 10:21:52 -0400 (EDT) Date: Fri, 7 Sep 2012 10:21:52 -0400 (EDT) From: Benjamin Kaduk To: "O. Hartmann" In-Reply-To: Message-ID: References: <5049B1C2.6030604@zedat.fu-berlin.de> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrCIsWRmVeSWpSXmKPExsUixCmqrMvC7BVgsOwem8WcNx+YLBadaGey +DvrD5MDs8e0zQfZPGZ8ms/icWr7QcYA5igum5TUnMyy1CJ9uwSujM3T/zIWfOSoaNzp1sA4 kb2LkZNDQsBE4sbhJhYIW0ziwr31bF2MXBxCAvsYJW5d3s0O4axnlNh46zUzhLOfSWLFhF/M IC1CAvUS2+4fZ+xi5OBgEdCSuLPYGCTMJqAiMfPNRjYQW0RAX+Jc02kwm1kgTqLp0w2wVmEB Z4n5S18xg7RyCgRK7JrmDhLmFbCXOLp+ItT0MokLU06ygtiiAjoSq/dPYYGoEZQ4OfMJC8RI S4l/a3+xTmAUnIUkNQtJagEj0ypG2ZTcKt3cxMyc4tRk3eLkxLy81CJdc73czBK91JTSTYyg wGV3UdnB2HxI6RCjAAejEg+vwVmPACHWxLLiytxDjJIcTEqivEx/PAOE+JLyUyozEosz4otK c1KLDzFKcDArifDuOwmU401JrKxKLcqHSUlzsCiJ815PuekvJJCeWJKanZpakFoEk5Xh4FCS 4H3B6BUgJFiUmp5akZaZU4KQZuLgBBnOAzT8PkgNb3FBYm5xZjpE/hSjopQ472OQhABIIqM0 D64XllheMYoDvSLM+xGkigeYlOC6XwENZgIaLPLMA2RwSSJCSqqB0XDx0j/Pd95LerUvnq1c 6Xr+kSA9C32eNdmt2bo8k+bFMyk9SKjsTIq4rneZuab6wnvdd5PlBV6/Ljgrob5uwbclz9rl FxZXztA96nnN106tqD5nVXrzvbcVYTrf3UWawh6d7T+WfHHBiZsXDsvce2W7i73Uf3Zw0ZHr Fz53Nq+4cH7p3VX285VYijMSDbWYi4oTAZLSKXEHAwAA Cc: Eitan Adler , Current FreeBSD Subject: Re: make: don't know how to make add-plist-buildinfo. Stop X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 14:22:03 -0000 On Fri, 7 Sep 2012, Eitan Adler wrote: > On 7 September 2012 04:35, O. Hartmann wrote: >> On all portupgrades, port installations I receive this error since today >> on FreeBSD 10.0-CURRENT amd64 r240150M: >> >> make: don't know how to make add-plist-buildinfo. Stop Did you try searching for the add-plist-buildinfo target? Grepping around in /usr/ports/Mk/* found me a definition of that target, conditional on PACKAGE_BUILDING being defined; doing so was an effective workaround until I managed to break my system more differently [1]. >> *** [reinstall] Error code 2 > > Fixed already. update your ports tree. Good to know that I won't have to fight this one again. -Ben Kaduk [1] Upgrading from an old -current past the set_rcvar() deprecation, so figured I'd just upgrade all ports. It seems that a post-pkg2ng pkg_delete -a has rendered /usr/sbin/pkg unwilling to bootstrap up, being "Unable to create local database!", though I've hacked this system up a bit in other ways so that may not be the real cause. More mail once I have a better grip on things. From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 15:10:03 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 23B611065673 for ; Fri, 7 Sep 2012 15:10:03 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) by mx1.freebsd.org (Postfix) with ESMTP id CBB7A8FC1B for ; Fri, 7 Sep 2012 15:10:02 +0000 (UTC) Received: from [192.168.0.6] (spaceball.home.andric.com [192.168.0.6]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id C0EE85C37; Fri, 7 Sep 2012 17:09:55 +0200 (CEST) Message-ID: <504A0E46.3010306@FreeBSD.org> Date: Fri, 07 Sep 2012 17:09:58 +0200 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20120905 Thunderbird/16.0 MIME-Version: 1.0 To: "O. Hartmann" References: <5049C13E.5010808@mail.zedat.fu-berlin.de> In-Reply-To: <5049C13E.5010808@mail.zedat.fu-berlin.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Current FreeBSD Subject: Re: FreeBSD 10.0-CURRENT: CLANG and port/clang weirdness! X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 15:10:03 -0000 On 2012-09-07 11:41, O. Hartmann wrote: > Building ports not explicitely enabling USE_GCC=4.6+ are considered > using the system's LLVM/CLANG, which is clang 3.2 in our installation > (FreeBSD 10.0-CURRENT #0 r240164), but since some ports require the > special ports devel/llvm and lang/clang, LLVM 3.1 and clang 3.1 get > installed and 3.1 is used instead the system's 3.2 whenever "clang", > "clang++" is invoked. Maybe a solution would be to use the same approach as with the gcc ports, namely installing the clang 3.1 executables into /usr/local/bin as clang-3.1, clang++-3.1 and clang-cpp-3.1. Then you could simply set CC=clang-3.1 CXX=clang++-3.1 CPP=clang-cpp-3.1 for the targets that require it. Brooks? :) > Following the WIKI at http://wiki.freebsd.org/BuildingFreeBSDWithClang > introduces the usage of > > CC=clang instead of CC=/usr/bin/clang > CXX=clang++ instead of CXX=/usr/bin/clang++ > CPP=clang-ccp instead of CPP=/usr/bin/clang-ccp > > Is this intended? Yes. During buildworld, in the cross-tools stage, a new compiler is built, and it is placed under ${WORLDTMP}, usually /usr/obj/usr/src/tmp. Afterwards, in the rest of the stages, the PATH is changed so executables from ${WORLDTMP} are preferred above those in the system directories. Therefore, if you set CC/CXX/CPP with an explicit path, this logic will not work, and your buildworld may have all kinds of trouble. I think there are several patches floating around to fix this, in various different ways. > Since I can not simply change the search patch - I need to have > /usr/local/bin before /usr/bin, is there a way to avoid this confusion? Yes, don't install the clang port, or do install it, but manually move the conflicting executables away, or delete them. This can't be fixed without fixing the clang ports to accept an option to change the compiler names into something non-conflicting. ... > My /etc/make.conf portion looks this: > > > > ## > ## CLANG > ## > .if !defined(NO_CLANG) > .if !defined(CC) || ${CC} == "cc" > CC= /usr/bin/clang > .endif > .if !defined(CXX) || ${CXX} == "c++" > CXX= /usr/bin/clang++ > .endif > .if !defined(CPP) || ${CPP} == "cpp" > CPP= /usr/bin/clang-cpp > .endif As said, putting an absolute path in these settings will defeat the logic in buildworld. Please don't do it, until there is a system in place to make this possible. (This is actually one of the stated goals for 10.0, to be able to specify even external toolchains for building world.) From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 15:11:04 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 802DA1065694 for ; Fri, 7 Sep 2012 15:11:04 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: from mail-wg0-f50.google.com (mail-wg0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 0D7888FC21 for ; Fri, 7 Sep 2012 15:11:03 +0000 (UTC) Received: by wgbds11 with SMTP id ds11so2445113wgb.31 for ; Fri, 07 Sep 2012 08:11:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=LiFf5B8LV0Xk6hGyd2Azgstk/Pcfz0lfrpsljs5s7/A=; b=0bYIeHYgE1U5Whicnobssw4/h7wfuLdYxA/TsxtlweRN99O/yiMvn8x4fXSgCX8oLB CySw7JjPrvNMIXislwNowmMKujvQUVGXm9P5Sd83Ghm7985O9Bxl3JobFKXkk/WETnh7 Dl0ir8UgSj7wOpYJgjR1Be1pQt6s5wGCR4vnZ3hQc7QJOaDep2dVxmtd/byMuB13gw8u Nr1vMIUQigkk7YSMxMNQZqRh83k5BsTGImrdXWnwXb7jiAmmBqQbL+90n6U8P7SmKY7/ sVN4mciLiR57C0SZ67OQA9ssiz6+QTxl7JFcgw9HwSHWMwrt9V+RneFgfxByKH+4PUSo vcdQ== MIME-Version: 1.0 Received: by 10.180.82.164 with SMTP id j4mr12888189wiy.18.1347030661996; Fri, 07 Sep 2012 08:11:01 -0700 (PDT) Received: by 10.194.0.145 with HTTP; Fri, 7 Sep 2012 08:11:01 -0700 (PDT) In-Reply-To: <20120904130039.GX33100@deviant.kiev.zoral.com.ua> References: <20120903124642.GI33100@deviant.kiev.zoral.com.ua> <20120904130039.GX33100@deviant.kiev.zoral.com.ua> Date: Fri, 7 Sep 2012 17:11:01 +0200 Message-ID: From: Svatopluk Kraus To: Konstantin Belousov Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-current@freebsd.org Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 15:11:04 -0000 On Tue, Sep 4, 2012 at 3:00 PM, Konstantin Belousov wrote: > On Tue, Sep 04, 2012 at 02:49:07PM +0200, Svatopluk Kraus wrote: >> On Mon, Sep 3, 2012 at 2:46 PM, Konstantin Belousov wrote: >> > On Mon, Sep 03, 2012 at 12:35:08PM +0200, Svatopluk Kraus wrote: >> >> Hi, >> >> >> >> I found out that while the running excecutables and a dynamic linker >> >> are protected against writing (ETXTBSY), the loaded shared libraries >> >> are not protected. The libraries are mapped by mmap() in dynamic >> >> linker (rtld) and there is no way how to set VV_TEXT flag on the >> >> libraries vnodes in mmap() code. >> >> >> >> In linux compability code \compat\linux\linux_misc.c, linux_uselib() >> >> sets VV_TEXT flags on a library vnode. In Solaris, MAP_TEXT flag >> >> exists which informs mmap() that the mapped region will be used >> >> primarily for executing instructions (for better MMU utilization). >> >> With these on mind, I propose to implement MAP_TEXT option in mmap() >> >> and in case that underlying object is a vnode, set VV_TEXT flag on it. >> >> >> >> I already have implemented it and with rtld map_object() patch it >> >> works fine for me (of course). The rtld patch looks easy, however I'm >> >> not sure about mmap patch. >> >> >> >> After some investigation, it looks that VV_TEXT once set on a vnode >> >> remains set until last reference on the vnode is left. So, I don't >> >> bother with VV_TEXT unset in munmap() to be consistent. The >> >> executables and dynamic linker are activated in kernel, so VV_TEXT is >> >> set before activation and cleared if something failed. Shared library >> >> activation is done in dynamic linker (i.e., in userland). It's done in >> >> steps and mmaping the library is one from them. So, I think that >> >> VV_TEXT can be set in mmap() just after everything is finished >> >> successfully. >> > This is right, the object reference counter is also used as >> > VV_TEXT counter. It is somewhat unaccurate, but in practice does >> > not cause issues. >> > >> >> >> >> The patch itself is implemented in vm_mmap_vnode(). If I want to set >> >> VV_TEXT flag on a vnode, I need an exclusive lock. In current code, >> >> the exclusive lock flag is (mis)used as a flag for >> >> vnode_pager_update_writecount() call. (I hope that I didn't miss >> >> something.) So, the patch is bigger slightly. >> >> >> >> I defined the MAP_TEXT flag in extented flags sections. However, I'm >> >> feeling the relation to MAP_STACK flag, but not sure if and when >> >> reserved flags (in other flags section) can be re-used. >> >> >> >> Svata >> >> >> >> >> >> Index: libexec/rtld-elf/map_object.c >> >> =================================================================== >> >> --- libexec/rtld-elf/map_object.c (revision 239770) >> >> +++ libexec/rtld-elf/map_object.c (working copy) >> >> @@ -199,7 +199,8 @@ >> >> data_prot = convert_prot(segs[i]->p_flags); >> >> data_flags = convert_flags(segs[i]->p_flags) | MAP_FIXED; >> >> if (mmap(data_addr, data_vlimit - data_vaddr, data_prot, >> >> - data_flags | MAP_PREFAULT_READ, fd, data_offset) == (caddr_t) -1) { >> >> + data_flags | MAP_PREFAULT_READ | MAP_TEXT, fd, data_offset) == >> >> + (caddr_t) -1) { >> > I am not sure that we shall mark all segments mappings with MAP_TEXT. >> > I understand the logic of the change, since we do not want data segment >> > to be changed under us. Still, having MAP_TEXT for non-text segments looks >> > strange. >> > >> >> I agree. However, only way how to recognize a text segment is an >> executable flag set. The new patch for map_object.c is following: >> >> Index: libexec/rtld-elf/map_object.c >> =================================================================== >> --- libexec/rtld-elf/map_object.c (revision 239770) >> +++ libexec/rtld-elf/map_object.c (working copy) >> @@ -442,5 +442,10 @@ >> */ >> if (!(elfflags & PF_W)) >> flags |= MAP_NOCORE; >> + /* >> + * Executable mappings are marked "MAP_TEXT". >> + */ >> + if (elfflags & PF_X) >> + flags |= MAP_TEXT; >> return flags; >> } >> >> >> >> _rtld_error("%s: mmap of data failed: %s", path, >> >> rtld_strerror(errno)); >> >> goto error1; >> >> Index: sys/vm/vm_mmap.c >> >> =================================================================== >> >> --- sys/vm/vm_mmap.c (revision 239770) >> >> +++ sys/vm/vm_mmap.c (working copy) >> >> @@ -1258,10 +1258,13 @@ >> >> struct mount *mp; >> >> struct ucred *cred; >> >> int error, flags, locktype, vfslocked; >> >> + int writeable_shared; >> >> >> >> mp = vp->v_mount; >> >> cred = td->td_ucred; >> >> - if ((*maxprotp & VM_PROT_WRITE) && (*flagsp & MAP_SHARED)) >> >> + flags = *flagsp; >> >> + writeable_shared = ((*maxprotp & VM_PROT_WRITE) && (flags & MAP_SHARED)); >> >> + if (writeable_shared || ((flags & MAP_TEXT) != 0)) >> >> locktype = LK_EXCLUSIVE; >> >> else >> >> locktype = LK_SHARED; >> >> @@ -1271,7 +1274,6 @@ >> >> return (error); >> >> } >> >> foff = *foffp; >> >> - flags = *flagsp; >> >> obj = vp->v_object; >> >> if (vp->v_type == VREG) { >> >> /* >> >> @@ -1294,7 +1296,7 @@ >> >> return (error); >> >> } >> >> } >> >> - if (locktype == LK_EXCLUSIVE) { >> >> + if (writeable_shared) { >> >> *writecounted = TRUE; >> >> vnode_pager_update_writecount(obj, 0, objsize); >> >> } >> >> @@ -1337,6 +1339,14 @@ >> >> error = ENOMEM; >> >> goto done; >> >> } >> >> + /* >> >> + * If MAP_TEXT is announced, set VV_TEXT so no one can write >> >> + * to the executable. >> >> + */ >> >> + if ((flags & MAP_TEXT) != 0) { >> >> + ASSERT_VOP_ELOCKED(vp, "vv_text"); >> >> + vp->v_vflag |= VV_TEXT; >> >> + } >> > I do not think we want to set VV_TEXT for device vnodes. >> > >> >> I agree too. However, my patch doesn't set VV_TEXT for device vnodes. >> Device vnodes never enter into patched part of code. > Hm, yes. > > Anyway, after thinking about the patch more, I see two issues: > > 1. You are setting VV_TEXT without checking v_writecount. This basically > nullifies the main reason for the patch, since existing writer can still > write or truncate the shared library after the mapping. You are right. I was thinking about it before. Now, when you pointed me to v_writecount, I found it in exec_check_permissions(). Thanks. So, executables activated in kernel are well protected. But the question is: for what, when shared libraries are avoided? > 2. I do not see what would prevent malicious local user from mmaping > arbitrary file readonly with MAP_TEXT, thus blocking any modifications > to the file. Note that this is not a problem for executables, because > kernel only sets VV_TEXT on executables if +x permission is set and > file is valid binary which kernel is able to execute. > > E.g. you might block log writes with VV_TEXT, or other user editing > session or whatever, having just read access to corresponding files. > > Am I wrong ? You are right again. I was too focused to my embeded single user boxes. Well, in general, setting VV_TEXT according to MAP_TEXT in mmap() is not secure. From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 15:12:39 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8DA481065672; Fri, 7 Sep 2012 15:12:39 +0000 (UTC) (envelope-from onwahe@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id D27288FC18; Fri, 7 Sep 2012 15:12:38 +0000 (UTC) Received: by weyx56 with SMTP id x56so2203850wey.13 for ; Fri, 07 Sep 2012 08:12:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=b8j8mgO0t4zvWmzpTPV+bXGWBeA+/Cn6vekRd72nUeQ=; b=CJhwM7Q2op95ya7xi5eqQSunfB0uBN84hZb+mOwa4VaRzFCfW+A/3bcMh9bLtljPWh +Q9KGxDlLuUE77GNBPeqWlEN7xnUgBuiyJrI9np0muL+z6DRGwQcHHSzQ64i79xhjp7q PQ8YzLzo0o4L6pashLMXtgBNx1jJjxAymFwdlhKIYpoR71iZJaSidJHKaTFCa9JppaoD ED5DqfBhhU/J7HgfyhX2NoX6Y8H0wwwHYv/JzHRwhm4ezNXOYiXxr88Nt9+c9DKFuhPN co1+Oy57ioOdF2z20XnM+p3Jj1+TLHIsHajRvhZXUhrUX3VhJrU/5Z/Nr24xUTUfXb1D xaWQ== MIME-Version: 1.0 Received: by 10.216.241.202 with SMTP id g52mr1191263wer.212.1347030757176; Fri, 07 Sep 2012 08:12:37 -0700 (PDT) Received: by 10.194.0.145 with HTTP; Fri, 7 Sep 2012 08:12:37 -0700 (PDT) In-Reply-To: <201209041200.27100.jhb@freebsd.org> References: <20120904130039.GX33100@deviant.kiev.zoral.com.ua> <201209041200.27100.jhb@freebsd.org> Date: Fri, 7 Sep 2012 17:12:37 +0200 Message-ID: From: Svatopluk Kraus To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: Konstantin Belousov , freebsd-current@freebsd.org Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 15:12:39 -0000 On Tue, Sep 4, 2012 at 6:00 PM, John Baldwin wrote: > On Tuesday, September 04, 2012 9:00:39 am Konstantin Belousov wrote: >> On Tue, Sep 04, 2012 at 02:49:07PM +0200, Svatopluk Kraus wrote: >> > On Mon, Sep 3, 2012 at 2:46 PM, Konstantin Belousov > wrote: >> > > On Mon, Sep 03, 2012 at 12:35:08PM +0200, Svatopluk Kraus wrote: >> > >> Hi, >> > >> >> > >> I found out that while the running excecutables and a dynamic linker >> > >> are protected against writing (ETXTBSY), the loaded shared libraries >> > >> are not protected. The libraries are mapped by mmap() in dynamic >> > >> linker (rtld) and there is no way how to set VV_TEXT flag on the >> > >> libraries vnodes in mmap() code. >> > >> >> > >> In linux compability code \compat\linux\linux_misc.c, linux_uselib() >> > >> sets VV_TEXT flags on a library vnode. In Solaris, MAP_TEXT flag >> > >> exists which informs mmap() that the mapped region will be used >> > >> primarily for executing instructions (for better MMU utilization). >> > >> With these on mind, I propose to implement MAP_TEXT option in mmap() >> > >> and in case that underlying object is a vnode, set VV_TEXT flag on it. >> > >> >> > >> I already have implemented it and with rtld map_object() patch it >> > >> works fine for me (of course). The rtld patch looks easy, however I'm >> > >> not sure about mmap patch. >> > >> >> > >> After some investigation, it looks that VV_TEXT once set on a vnode >> > >> remains set until last reference on the vnode is left. So, I don't >> > >> bother with VV_TEXT unset in munmap() to be consistent. The >> > >> executables and dynamic linker are activated in kernel, so VV_TEXT is >> > >> set before activation and cleared if something failed. Shared library >> > >> activation is done in dynamic linker (i.e., in userland). It's done in >> > >> steps and mmaping the library is one from them. So, I think that >> > >> VV_TEXT can be set in mmap() just after everything is finished >> > >> successfully. >> > > This is right, the object reference counter is also used as >> > > VV_TEXT counter. It is somewhat unaccurate, but in practice does >> > > not cause issues. >> > > >> > >> >> > >> The patch itself is implemented in vm_mmap_vnode(). If I want to set >> > >> VV_TEXT flag on a vnode, I need an exclusive lock. In current code, >> > >> the exclusive lock flag is (mis)used as a flag for >> > >> vnode_pager_update_writecount() call. (I hope that I didn't miss >> > >> something.) So, the patch is bigger slightly. >> > >> >> > >> I defined the MAP_TEXT flag in extented flags sections. However, I'm >> > >> feeling the relation to MAP_STACK flag, but not sure if and when >> > >> reserved flags (in other flags section) can be re-used. >> > >> >> > >> Svata >> > >> >> > >> >> > >> Index: libexec/rtld-elf/map_object.c >> > >> =================================================================== >> > >> --- libexec/rtld-elf/map_object.c (revision 239770) >> > >> +++ libexec/rtld-elf/map_object.c (working copy) >> > >> @@ -199,7 +199,8 @@ >> > >> data_prot = convert_prot(segs[i]->p_flags); >> > >> data_flags = convert_flags(segs[i]->p_flags) | MAP_FIXED; >> > >> if (mmap(data_addr, data_vlimit - data_vaddr, data_prot, >> > >> - data_flags | MAP_PREFAULT_READ, fd, data_offset) == (caddr_t) > -1) { >> > >> + data_flags | MAP_PREFAULT_READ | MAP_TEXT, fd, data_offset) == >> > >> + (caddr_t) -1) { >> > > I am not sure that we shall mark all segments mappings with MAP_TEXT. >> > > I understand the logic of the change, since we do not want data segment >> > > to be changed under us. Still, having MAP_TEXT for non-text segments > looks >> > > strange. >> > > >> > >> > I agree. However, only way how to recognize a text segment is an >> > executable flag set. The new patch for map_object.c is following: >> > >> > Index: libexec/rtld-elf/map_object.c >> > =================================================================== >> > --- libexec/rtld-elf/map_object.c (revision 239770) >> > +++ libexec/rtld-elf/map_object.c (working copy) >> > @@ -442,5 +442,10 @@ >> > */ >> > if (!(elfflags & PF_W)) >> > flags |= MAP_NOCORE; >> > + /* >> > + * Executable mappings are marked "MAP_TEXT". >> > + */ >> > + if (elfflags & PF_X) >> > + flags |= MAP_TEXT; >> > return flags; >> > } >> > >> > >> > >> _rtld_error("%s: mmap of data failed: %s", path, >> > >> rtld_strerror(errno)); >> > >> goto error1; >> > >> Index: sys/vm/vm_mmap.c >> > >> =================================================================== >> > >> --- sys/vm/vm_mmap.c (revision 239770) >> > >> +++ sys/vm/vm_mmap.c (working copy) >> > >> @@ -1258,10 +1258,13 @@ >> > >> struct mount *mp; >> > >> struct ucred *cred; >> > >> int error, flags, locktype, vfslocked; >> > >> + int writeable_shared; >> > >> >> > >> mp = vp->v_mount; >> > >> cred = td->td_ucred; >> > >> - if ((*maxprotp & VM_PROT_WRITE) && (*flagsp & MAP_SHARED)) >> > >> + flags = *flagsp; >> > >> + writeable_shared = ((*maxprotp & VM_PROT_WRITE) && (flags & > MAP_SHARED)); >> > >> + if (writeable_shared || ((flags & MAP_TEXT) != 0)) >> > >> locktype = LK_EXCLUSIVE; >> > >> else >> > >> locktype = LK_SHARED; >> > >> @@ -1271,7 +1274,6 @@ >> > >> return (error); >> > >> } >> > >> foff = *foffp; >> > >> - flags = *flagsp; >> > >> obj = vp->v_object; >> > >> if (vp->v_type == VREG) { >> > >> /* >> > >> @@ -1294,7 +1296,7 @@ >> > >> return (error); >> > >> } >> > >> } >> > >> - if (locktype == LK_EXCLUSIVE) { >> > >> + if (writeable_shared) { >> > >> *writecounted = TRUE; >> > >> vnode_pager_update_writecount(obj, 0, objsize); >> > >> } >> > >> @@ -1337,6 +1339,14 @@ >> > >> error = ENOMEM; >> > >> goto done; >> > >> } >> > >> + /* >> > >> + * If MAP_TEXT is announced, set VV_TEXT so no one can write >> > >> + * to the executable. >> > >> + */ >> > >> + if ((flags & MAP_TEXT) != 0) { >> > >> + ASSERT_VOP_ELOCKED(vp, "vv_text"); >> > >> + vp->v_vflag |= VV_TEXT; >> > >> + } >> > > I do not think we want to set VV_TEXT for device vnodes. >> > > >> > >> > I agree too. However, my patch doesn't set VV_TEXT for device vnodes. >> > Device vnodes never enter into patched part of code. >> Hm, yes. >> >> Anyway, after thinking about the patch more, I see two issues: >> >> 1. You are setting VV_TEXT without checking v_writecount. This basically >> nullifies the main reason for the patch, since existing writer can still >> write or truncate the shared library after the mapping. >> >> 2. I do not see what would prevent malicious local user from mmaping >> arbitrary file readonly with MAP_TEXT, thus blocking any modifications >> to the file. Note that this is not a problem for executables, because >> kernel only sets VV_TEXT on executables if +x permission is set and >> file is valid binary which kernel is able to execute. >> >> E.g. you might block log writes with VV_TEXT, or other user editing >> session or whatever, having just read access to corresponding files. >> >> Am I wrong ? > > Hmm, I do think 2) is a bit of a show-stopper. I do wonder why one needs > MAP_TEXT at all or if you could key this off of mmap() with PROT_EXEC? > Do we require +x permissions for PROT_EXEC? No, it seems we only require > a file opened with FREAD. Hmm, perhaps rtld could open a separate fd for > PROT_EXEC mappings that used O_EXEC and mmap()'ing an O_EXEC fd could enable > VV_TEXT? That would require a file to have +x permisson for an mmap() to > enable VV_TEXT. It would also make MAP_TEXT unneeded. It sounds good for me. I will try to patch it this way. However, do you think that will be acceptable to set +x permission to shared libraries in general? Svata > > -- > John Baldwin From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 16:02:16 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6077B106564A; Fri, 7 Sep 2012 16:02:16 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id B7F648FC20; Fri, 7 Sep 2012 16:02:14 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q87G2KZY016585; Fri, 7 Sep 2012 19:02:20 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q87G2853071468; Fri, 7 Sep 2012 19:02:08 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q87G28w8071467; Fri, 7 Sep 2012 19:02:08 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 7 Sep 2012 19:02:08 +0300 From: Konstantin Belousov To: Svatopluk Kraus Message-ID: <20120907160208.GZ33100@deviant.kiev.zoral.com.ua> References: <20120904130039.GX33100@deviant.kiev.zoral.com.ua> <201209041200.27100.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="XAHTXMpTldIJKQ/i" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-current@freebsd.org Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 16:02:16 -0000 --XAHTXMpTldIJKQ/i Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Sep 07, 2012 at 05:12:37PM +0200, Svatopluk Kraus wrote: > On Tue, Sep 4, 2012 at 6:00 PM, John Baldwin wrote: > > On Tuesday, September 04, 2012 9:00:39 am Konstantin Belousov wrote: > >> On Tue, Sep 04, 2012 at 02:49:07PM +0200, Svatopluk Kraus wrote: > >> > On Mon, Sep 3, 2012 at 2:46 PM, Konstantin Belousov > > wrote: > >> > > On Mon, Sep 03, 2012 at 12:35:08PM +0200, Svatopluk Kraus wrote: > >> > >> Hi, > >> > >> > >> > >> I found out that while the running excecutables and a dynamic l= inker > >> > >> are protected against writing (ETXTBSY), the loaded shared librar= ies > >> > >> are not protected. The libraries are mapped by mmap() in dynamic > >> > >> linker (rtld) and there is no way how to set VV_TEXT flag on the > >> > >> libraries vnodes in mmap() code. > >> > >> > >> > >> In linux compability code \compat\linux\linux_misc.c, linux_use= lib() > >> > >> sets VV_TEXT flags on a library vnode. In Solaris, MAP_TEXT flag > >> > >> exists which informs mmap() that the mapped region will be used > >> > >> primarily for executing instructions (for better MMU utilization). > >> > >> With these on mind, I propose to implement MAP_TEXT option in mma= p() > >> > >> and in case that underlying object is a vnode, set VV_TEXT flag o= n it. > >> > >> > >> > >> I already have implemented it and with rtld map_object() patch = it > >> > >> works fine for me (of course). The rtld patch looks easy, however= I'm > >> > >> not sure about mmap patch. > >> > >> > >> > >> After some investigation, it looks that VV_TEXT once set on a v= node > >> > >> remains set until last reference on the vnode is left. So, I don't > >> > >> bother with VV_TEXT unset in munmap() to be consistent. The > >> > >> executables and dynamic linker are activated in kernel, so VV_TEX= T is > >> > >> set before activation and cleared if something failed. Shared lib= rary > >> > >> activation is done in dynamic linker (i.e., in userland). It's do= ne in > >> > >> steps and mmaping the library is one from them. So, I think that > >> > >> VV_TEXT can be set in mmap() just after everything is finished > >> > >> successfully. > >> > > This is right, the object reference counter is also used as > >> > > VV_TEXT counter. It is somewhat unaccurate, but in practice does > >> > > not cause issues. > >> > > > >> > >> > >> > >> The patch itself is implemented in vm_mmap_vnode(). If I want t= o set > >> > >> VV_TEXT flag on a vnode, I need an exclusive lock. In current cod= e, > >> > >> the exclusive lock flag is (mis)used as a flag for > >> > >> vnode_pager_update_writecount() call. (I hope that I didn't miss > >> > >> something.) So, the patch is bigger slightly. > >> > >> > >> > >> I defined the MAP_TEXT flag in extented flags sections. However= , I'm > >> > >> feeling the relation to MAP_STACK flag, but not sure if and when > >> > >> reserved flags (in other flags section) can be re-used. > >> > >> > >> > >> Svata > >> > >> > >> > >> > >> > >> Index: libexec/rtld-elf/map_object.c > >> > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > >> > >> --- libexec/rtld-elf/map_object.c (revision 239770) > >> > >> +++ libexec/rtld-elf/map_object.c (working copy) > >> > >> @@ -199,7 +199,8 @@ > >> > >> data_prot =3D convert_prot(segs[i]->p_flags); > >> > >> data_flags =3D convert_flags(segs[i]->p_flags) | MAP_FIXED; > >> > >> if (mmap(data_addr, data_vlimit - data_vaddr, data_prot, > >> > >> - data_flags | MAP_PREFAULT_READ, fd, data_offset) =3D=3D (= caddr_t) > > -1) { > >> > >> + data_flags | MAP_PREFAULT_READ | MAP_TEXT, fd, data_offse= t) =3D=3D > >> > >> + (caddr_t) -1) { > >> > > I am not sure that we shall mark all segments mappings with MAP_TE= XT. > >> > > I understand the logic of the change, since we do not want data se= gment > >> > > to be changed under us. Still, having MAP_TEXT for non-text segmen= ts > > looks > >> > > strange. > >> > > > >> > > >> > I agree. However, only way how to recognize a text segment is an > >> > executable flag set. The new patch for map_object.c is following: > >> > > >> > Index: libexec/rtld-elf/map_object.c > >> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > >> > --- libexec/rtld-elf/map_object.c (revision 239770) > >> > +++ libexec/rtld-elf/map_object.c (working copy) > >> > @@ -442,5 +442,10 @@ > >> > */ > >> > if (!(elfflags & PF_W)) > >> > flags |=3D MAP_NOCORE; > >> > + /* > >> > + * Executable mappings are marked "MAP_TEXT". > >> > + */ > >> > + if (elfflags & PF_X) > >> > + flags |=3D MAP_TEXT; > >> > return flags; > >> > } > >> > > >> > > >> > >> _rtld_error("%s: mmap of data failed: %s", path, > >> > >> rtld_strerror(errno)); > >> > >> goto error1; > >> > >> Index: sys/vm/vm_mmap.c > >> > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > >> > >> --- sys/vm/vm_mmap.c (revision 239770) > >> > >> +++ sys/vm/vm_mmap.c (working copy) > >> > >> @@ -1258,10 +1258,13 @@ > >> > >> struct mount *mp; > >> > >> struct ucred *cred; > >> > >> int error, flags, locktype, vfslocked; > >> > >> + int writeable_shared; > >> > >> > >> > >> mp =3D vp->v_mount; > >> > >> cred =3D td->td_ucred; > >> > >> - if ((*maxprotp & VM_PROT_WRITE) && (*flagsp & MAP_SHARED)) > >> > >> + flags =3D *flagsp; > >> > >> + writeable_shared =3D ((*maxprotp & VM_PROT_WRITE) && (flags= & > > MAP_SHARED)); > >> > >> + if (writeable_shared || ((flags & MAP_TEXT) !=3D 0)) > >> > >> locktype =3D LK_EXCLUSIVE; > >> > >> else > >> > >> locktype =3D LK_SHARED; > >> > >> @@ -1271,7 +1274,6 @@ > >> > >> return (error); > >> > >> } > >> > >> foff =3D *foffp; > >> > >> - flags =3D *flagsp; > >> > >> obj =3D vp->v_object; > >> > >> if (vp->v_type =3D=3D VREG) { > >> > >> /* > >> > >> @@ -1294,7 +1296,7 @@ > >> > >> return (error); > >> > >> } > >> > >> } > >> > >> - if (locktype =3D=3D LK_EXCLUSIVE) { > >> > >> + if (writeable_shared) { > >> > >> *writecounted =3D TRUE; > >> > >> vnode_pager_update_writecount(obj, 0, objsi= ze); > >> > >> } > >> > >> @@ -1337,6 +1339,14 @@ > >> > >> error =3D ENOMEM; > >> > >> goto done; > >> > >> } > >> > >> + /* > >> > >> + * If MAP_TEXT is announced, set VV_TEXT so no one can write > >> > >> + * to the executable. > >> > >> + */ > >> > >> + if ((flags & MAP_TEXT) !=3D 0) { > >> > >> + ASSERT_VOP_ELOCKED(vp, "vv_text"); > >> > >> + vp->v_vflag |=3D VV_TEXT; > >> > >> + } > >> > > I do not think we want to set VV_TEXT for device vnodes. > >> > > > >> > > >> > I agree too. However, my patch doesn't set VV_TEXT for device vnodes. > >> > Device vnodes never enter into patched part of code. > >> Hm, yes. > >> > >> Anyway, after thinking about the patch more, I see two issues: > >> > >> 1. You are setting VV_TEXT without checking v_writecount. This basical= ly > >> nullifies the main reason for the patch, since existing writer can = still > >> write or truncate the shared library after the mapping. > >> > >> 2. I do not see what would prevent malicious local user from mmaping > >> arbitrary file readonly with MAP_TEXT, thus blocking any modificati= ons > >> to the file. Note that this is not a problem for executables, becau= se > >> kernel only sets VV_TEXT on executables if +x permission is set and > >> file is valid binary which kernel is able to execute. > >> > >> E.g. you might block log writes with VV_TEXT, or other user editing > >> session or whatever, having just read access to corresponding files. > >> > >> Am I wrong ? > > > > Hmm, I do think 2) is a bit of a show-stopper. I do wonder why one nee= ds > > MAP_TEXT at all or if you could key this off of mmap() with PROT_EXEC? > > Do we require +x permissions for PROT_EXEC? No, it seems we only requi= re > > a file opened with FREAD. Hmm, perhaps rtld could open a separate fd f= or > > PROT_EXEC mappings that used O_EXEC and mmap()'ing an O_EXEC fd could e= nable > > VV_TEXT? That would require a file to have +x permisson for an mmap() = to > > enable VV_TEXT. It would also make MAP_TEXT unneeded. >=20 > It sounds good for me. I will try to patch it this way. However, do > you think that will be acceptable to set +x permission to shared > libraries in general? Setting +x on shared libraries can be done. But setting VV_TEXT for such mappings is definitely non-standard behaviour, that could cause locking surprises for unaware system administrator. The issuw would be very hard to diagnose. --XAHTXMpTldIJKQ/i Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlBKGn8ACgkQC3+MBN1Mb4gkSwCghehBoaSb/w0YPJv4r6hg4jC3 lJ4An16tu/g9+BK71WiUBuqGGPqZVc8q =ZoO3 -----END PGP SIGNATURE----- --XAHTXMpTldIJKQ/i-- From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 16:24:33 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5AE7F1065670 for ; Fri, 7 Sep 2012 16:24:33 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 2DDF38FC14 for ; Fri, 7 Sep 2012 16:24:33 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 80395B978; Fri, 7 Sep 2012 12:24:32 -0400 (EDT) From: John Baldwin To: Konstantin Belousov Date: Fri, 7 Sep 2012 12:21:37 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) References: <20120907160208.GZ33100@deviant.kiev.zoral.com.ua> In-Reply-To: <20120907160208.GZ33100@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201209071221.37409.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 07 Sep 2012 12:24:32 -0400 (EDT) Cc: freebsd-current@freebsd.org, Svatopluk Kraus Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 16:24:33 -0000 On Friday, September 07, 2012 12:02:08 pm Konstantin Belousov wrote: > On Fri, Sep 07, 2012 at 05:12:37PM +0200, Svatopluk Kraus wrote: > > On Tue, Sep 4, 2012 at 6:00 PM, John Baldwin wrote: > > > On Tuesday, September 04, 2012 9:00:39 am Konstantin Belousov wrote: > > >> 2. I do not see what would prevent malicious local user from mmaping > > >> arbitrary file readonly with MAP_TEXT, thus blocking any modifications > > >> to the file. Note that this is not a problem for executables, because > > >> kernel only sets VV_TEXT on executables if +x permission is set and > > >> file is valid binary which kernel is able to execute. > > >> > > >> E.g. you might block log writes with VV_TEXT, or other user editing > > >> session or whatever, having just read access to corresponding files. > > >> > > >> Am I wrong ? > > > > > > Hmm, I do think 2) is a bit of a show-stopper. I do wonder why one needs > > > MAP_TEXT at all or if you could key this off of mmap() with PROT_EXEC? > > > Do we require +x permissions for PROT_EXEC? No, it seems we only require > > > a file opened with FREAD. Hmm, perhaps rtld could open a separate fd for > > > PROT_EXEC mappings that used O_EXEC and mmap()'ing an O_EXEC fd could enable > > > VV_TEXT? That would require a file to have +x permisson for an mmap() to > > > enable VV_TEXT. It would also make MAP_TEXT unneeded. > > > > It sounds good for me. I will try to patch it this way. However, do > > you think that will be acceptable to set +x permission to shared > > libraries in general? Shared libraries have +x by default already. You could make rtld fall back to using the O_RDONLY fd if it can't open an O_EXEC fd in which case you don't get the VV_TEXT protection, but rtld would be backwards compatible. > Setting +x on shared libraries can be done. But setting VV_TEXT for > such mappings is definitely non-standard behaviour, that could cause > locking surprises for unaware system administrator. The issuw would be > very hard to diagnose. I'm not sure it would be that obscure. It's the same as getting an error that a binary is busy. In that case you would resort to 'ps' to find the offending process. For this case (a shared library being busy), you could look at the output of 'procstat -af' to find which processes have executable mappings of the library. -- John Baldwin From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 16:42:26 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2011C106564A; Fri, 7 Sep 2012 16:42:26 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id A96AC8FC08; Fri, 7 Sep 2012 16:42:25 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q87GgUvZ020469; Fri, 7 Sep 2012 19:42:30 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q87GgIVw071646; Fri, 7 Sep 2012 19:42:18 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q87GgIGU071645; Fri, 7 Sep 2012 19:42:18 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 7 Sep 2012 19:42:18 +0300 From: Konstantin Belousov To: John Baldwin Message-ID: <20120907164218.GB33100@deviant.kiev.zoral.com.ua> References: <20120907160208.GZ33100@deviant.kiev.zoral.com.ua> <201209071221.37409.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="QW87tl/UEAXcMuUT" Content-Disposition: inline In-Reply-To: <201209071221.37409.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-current@freebsd.org, Svatopluk Kraus Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 16:42:26 -0000 --QW87tl/UEAXcMuUT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Sep 07, 2012 at 12:21:37PM -0400, John Baldwin wrote: > On Friday, September 07, 2012 12:02:08 pm Konstantin Belousov wrote: > > On Fri, Sep 07, 2012 at 05:12:37PM +0200, Svatopluk Kraus wrote: > > > On Tue, Sep 4, 2012 at 6:00 PM, John Baldwin wrote: > > > > On Tuesday, September 04, 2012 9:00:39 am Konstantin Belousov wrote: > > > >> 2. I do not see what would prevent malicious local user from mmapi= ng > > > >> arbitrary file readonly with MAP_TEXT, thus blocking any modifi= cations > > > >> to the file. Note that this is not a problem for executables, b= ecause > > > >> kernel only sets VV_TEXT on executables if +x permission is set= and > > > >> file is valid binary which kernel is able to execute. > > > >> > > > >> E.g. you might block log writes with VV_TEXT, or other user edi= ting > > > >> session or whatever, having just read access to corresponding f= iles. > > > >> > > > >> Am I wrong ? > > > > > > > > Hmm, I do think 2) is a bit of a show-stopper. I do wonder why one= needs > > > > MAP_TEXT at all or if you could key this off of mmap() with PROT_EX= EC? > > > > Do we require +x permissions for PROT_EXEC? No, it seems we only r= equire > > > > a file opened with FREAD. Hmm, perhaps rtld could open a separate = fd for > > > > PROT_EXEC mappings that used O_EXEC and mmap()'ing an O_EXEC fd cou= ld enable > > > > VV_TEXT? That would require a file to have +x permisson for an mma= p() to > > > > enable VV_TEXT. It would also make MAP_TEXT unneeded. > > >=20 > > > It sounds good for me. I will try to patch it this way. However, do > > > you think that will be acceptable to set +x permission to shared > > > libraries in general? >=20 > Shared libraries have +x by default already. You could make rtld fall ba= ck > to using the O_RDONLY fd if it can't open an O_EXEC fd in which case you = don't > get the VV_TEXT protection, but rtld would be backwards compatible. Where is it ? On fresh stable/9 machine, I have in /lib: -r--r--r-- 1 root wheel 7216 Dec 3 2011 libipx.so.5 -r--r--r-- 1 root wheel 19800 Jun 28 18:33 libjail.so.1 -r--r--r-- 1 root wheel 13752 Jun 28 18:33 libkiconv.so.4 =2E.. >=20 > > Setting +x on shared libraries can be done. But setting VV_TEXT for > > such mappings is definitely non-standard behaviour, that could cause > > locking surprises for unaware system administrator. The issuw would be > > very hard to diagnose. >=20 > I'm not sure it would be that obscure. It's the same as getting an error= that > a binary is busy. In that case you would resort to 'ps' to find the offe= nding > process. For this case (a shared library being busy), you could look at = the > output of 'procstat -af' to find which processes have executable mappings= of > the library. I much more worry about rtld reporting 'shared library is busy'. It is fine to not be able to write to mapped library. Rtld errors are usually quite worrying for users, and they just do not understand them. --QW87tl/UEAXcMuUT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlBKI+oACgkQC3+MBN1Mb4gJewCfZxKgKh/5rW+MP6kY04yvRYtD 1bMAn27o3TOvKlOoOzKLt/SzsL9l0kXF =FjLa -----END PGP SIGNATURE----- --QW87tl/UEAXcMuUT-- From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 15:46:17 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED64F106564A; Fri, 7 Sep 2012 15:46:16 +0000 (UTC) (envelope-from ohartman@mail.zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id 92A918FC15; Fri, 7 Sep 2012 15:46:16 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.69) with esmtp (envelope-from ) id <1TA0l0-0007ZH-F1>; Fri, 07 Sep 2012 17:46:10 +0200 Received: from telesto.geoinf.fu-berlin.de ([130.133.86.198]) by inpost2.zedat.fu-berlin.de (Exim 4.69) with esmtpsa (envelope-from ) id <1TA0l0-0006m1-BT>; Fri, 07 Sep 2012 17:46:10 +0200 Message-ID: <504A16BA.7030407@mail.zedat.fu-berlin.de> Date: Fri, 07 Sep 2012 17:46:02 +0200 From: "O. Hartmann" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120810 Thunderbird/14.0 MIME-Version: 1.0 To: Dimitry Andric References: <5049C13E.5010808@mail.zedat.fu-berlin.de> <504A0E46.3010306@FreeBSD.org> In-Reply-To: <504A0E46.3010306@FreeBSD.org> X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig77A406738018B61D522F186A" X-Originating-IP: 130.133.86.198 X-Mailman-Approved-At: Fri, 07 Sep 2012 16:50:12 +0000 Cc: Current FreeBSD Subject: Re: FreeBSD 10.0-CURRENT: CLANG and port/clang weirdness! X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 15:46:17 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig77A406738018B61D522F186A Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 09/07/12 17:09, Dimitry Andric wrote: > On 2012-09-07 11:41, O. Hartmann wrote: >> Building ports not explicitely enabling USE_GCC=3D4.6+ are considered >> using the system's LLVM/CLANG, which is clang 3.2 in our installation >> (FreeBSD 10.0-CURRENT #0 r240164), but since some ports require the >> special ports devel/llvm and lang/clang, LLVM 3.1 and clang 3.1 get >> installed and 3.1 is used instead the system's 3.2 whenever "clang", >> "clang++" is invoked. >=20 > Maybe a solution would be to use the same approach as with the gcc > ports, namely installing the clang 3.1 executables into /usr/local/bin > as clang-3.1, clang++-3.1 and clang-cpp-3.1. Then you could simply set= >=20 > CC=3Dclang-3.1 > CXX=3Dclang++-3.1 > CPP=3Dclang-cpp-3.1 >=20 > for the targets that require it. Brooks? :) I would appreciate such an approach, since it would be consistent with with GCC, as you stated. >=20 >=20 >> Following the WIKI at http://wiki.freebsd.org/BuildingFreeBSDWithClang= >> introduces the usage of >> >> CC=3Dclang instead of CC=3D/usr/bin/clang >> CXX=3Dclang++ instead of CXX=3D/usr/bin/clang++ >> CPP=3Dclang-ccp instead of CPP=3D/usr/bin/clang-ccp >> >> Is this intended? >=20 > Yes. During buildworld, in the cross-tools stage, a new compiler is > built, and it is placed under ${WORLDTMP}, usually /usr/obj/usr/src/tmp= =2E > Afterwards, in the rest of the stages, the PATH is changed so > executables from ${WORLDTMP} are preferred above those in the system > directories. >=20 > Therefore, if you set CC/CXX/CPP with an explicit path, this logic will= > not work, and your buildworld may have all kinds of trouble. I think > there are several patches floating around to fix this, in various > different ways. Understood. >=20 >=20 >> Since I can not simply change the search patch - I need to have >> /usr/local/bin before /usr/bin, is there a way to avoid this confusion= ? >=20 > Yes, don't install the clang port, or do install it, but manually move > the conflicting executables away, or delete them. This can't be fixed > without fixing the clang ports to accept an option to change the > compiler names into something non-conflicting. The LibreOffice package doesn't compile with the system's CLANG, so it is installed whenever LibreOffice is installed. We have on all workstations running FreeBSD LibreOffice installed and so devel/llvm and lang/clang get installed, as far as I know. >=20 >=20 > ... >> My /etc/make.conf portion looks this: >> >> >> >> ## >> ## CLANG >> ## >> .if !defined(NO_CLANG) >> .if !defined(CC) || ${CC} =3D=3D "cc" >> CC=3D /usr/bin/clang >> .endif >> .if !defined(CXX) || ${CXX} =3D=3D "c++" >> CXX=3D /usr/bin/clang++ >> .endif >> .if !defined(CPP) || ${CPP} =3D=3D "cpp" >> CPP=3D /usr/bin/clang-cpp >> .endif >=20 > As said, putting an absolute path in these settings will defeat the > logic in buildworld. Please don't do it, until there is a system in > place to make this possible. (This is actually one of the stated goals= > for 10.0, to be able to specify even external toolchains for building > world.) Changed back, so hopefully no danger to pollute the list with more questions about failures ;-) Since I'm using on most boxes 10.0, where can I read more about the new toolchain approach? And by the way - is there a way to have also LLVM installed with the base system by a knob like "WITH_LLVM"? This would avoid the same confusion as mentioned above with CLANG when it comes to LLVM dependencies (I play around with some OpenCL libs (pocl), which seems to need LLVM port installed). Thanks for your response, regards Oliver --------------enig77A406738018B61D522F186A Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBAgAGBQJQShbCAAoJEOgBcD7A/5N81qMIALE9ZuuAFslBSTcGStfJhXpz mG7nntPphLGmu/iaPerAFjkPRj+0y0ChAf7OGtPqnfmyg4tq1RzWKWOY7tRNXWfw 4c9okFUiV6+j/TAmiQ2/qIIONTSerrqP6f+DoNDYWKG44wz9r7jMKgzfmejJFPb8 M1r9kZMPeOAwMDq5LWWgHpRV5bxz0PwTrRU/xgfk50eslFh8yJaA2EJFvcJefCUt f4o6eibklK4fM/ChCRxjPbRK/nfJk7M1FMB/kvWnklkOVglVsrEu7iG9AHdCA0sn cnC8FRnIFXufDzuQe+bshsTfLDd+x/oGVQ9PenZMra9tHWb9BNPTZ1VGteAynDw= =m9G3 -----END PGP SIGNATURE----- --------------enig77A406738018B61D522F186A-- From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 17:07:36 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76C31106564A; Fri, 7 Sep 2012 17:07:36 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (lor.one-eyed-alien.net [69.66.77.232]) by mx1.freebsd.org (Postfix) with ESMTP id A2C7A8FC12; Fri, 7 Sep 2012 17:07:34 +0000 (UTC) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.5/8.14.5) with ESMTP id q87H7SEE035227; Fri, 7 Sep 2012 12:07:28 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.5/8.14.5/Submit) id q87H7Sfq035226; Fri, 7 Sep 2012 12:07:28 -0500 (CDT) (envelope-from brooks) Date: Fri, 7 Sep 2012 12:07:28 -0500 From: Brooks Davis To: "O. Hartmann" Message-ID: <20120907170728.GB28113@lor.one-eyed-alien.net> References: <5049C13E.5010808@mail.zedat.fu-berlin.de> <504A0E46.3010306@FreeBSD.org> <504A16BA.7030407@mail.zedat.fu-berlin.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="vtzGhvizbBRQ85DL" Content-Disposition: inline In-Reply-To: <504A16BA.7030407@mail.zedat.fu-berlin.de> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Current FreeBSD , Dimitry Andric Subject: Re: FreeBSD 10.0-CURRENT: CLANG and port/clang weirdness! X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 17:07:36 -0000 --vtzGhvizbBRQ85DL Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Sep 07, 2012 at 05:46:02PM +0200, O. Hartmann wrote: > On 09/07/12 17:09, Dimitry Andric wrote: > > On 2012-09-07 11:41, O. Hartmann wrote: > >> Building ports not explicitely enabling USE_GCC=3D4.6+ are considered > >> using the system's LLVM/CLANG, which is clang 3.2 in our installation > >> (FreeBSD 10.0-CURRENT #0 r240164), but since some ports require the > >> special ports devel/llvm and lang/clang, LLVM 3.1 and clang 3.1 get > >> installed and 3.1 is used instead the system's 3.2 whenever "clang", > >> "clang++" is invoked. > >=20 > > Maybe a solution would be to use the same approach as with the gcc > > ports, namely installing the clang 3.1 executables into /usr/local/bin > > as clang-3.1, clang++-3.1 and clang-cpp-3.1. Then you could simply set > >=20 > > CC=3Dclang-3.1 > > CXX=3Dclang++-3.1 > > CPP=3Dclang-cpp-3.1 > >=20 > > for the targets that require it. Brooks? :) >=20 > I would appreciate such an approach, since it would be consistent with > with GCC, as you stated. I'd like to do this, but it doesn't look like it will be easy. There appears to be no support in the llvm build system for it. :( > >> Following the WIKI at http://wiki.freebsd.org/BuildingFreeBSDWithClang > >> introduces the usage of > >> > >> CC=3Dclang instead of CC=3D/usr/bin/clang > >> CXX=3Dclang++ instead of CXX=3D/usr/bin/clang++ > >> CPP=3Dclang-ccp instead of CPP=3D/usr/bin/clang-ccp > >> > >> Is this intended? > >=20 > > Yes. During buildworld, in the cross-tools stage, a new compiler is > > built, and it is placed under ${WORLDTMP}, usually /usr/obj/usr/src/tmp. > > Afterwards, in the rest of the stages, the PATH is changed so > > executables from ${WORLDTMP} are preferred above those in the system > > directories. > >=20 > > Therefore, if you set CC/CXX/CPP with an explicit path, this logic will > > not work, and your buildworld may have all kinds of trouble. I think > > there are several patches floating around to fix this, in various > > different ways. >=20 > Understood. FWIW, picking up clang etc from /usr/local should be mostly harmless during the early build stage. You're actual world will be built with the cross clang. > Since I'm using on most boxes 10.0, where can I read more about the new > toolchain approach? Discussions will take place on the toolchain@ list. I'll be posting some writeups soon. > And by the way - is there a way to have also LLVM installed with the > base system by a knob like "WITH_LLVM"? This would avoid the same > confusion as mentioned above with CLANG when it comes to LLVM > dependencies (I play around with some OpenCL libs (pocl), which seems to > need LLVM port installed). You're probably looking for WITH_CLANG_EXTRAS. -- Brooks >=20 > Thanks for your response, >=20 > regards >=20 > Oliver >=20 >=20 --vtzGhvizbBRQ85DL Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iD8DBQFQSinPXY6L6fI4GtQRAoNDAKC8pWHN2AHPMikAanTYkGdmSnb2kgCfbtul i1AbdZPVqXjYv9dI9+/yUBE= =8jRa -----END PGP SIGNATURE----- --vtzGhvizbBRQ85DL-- From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 18:19:25 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2EC18106566B for ; Fri, 7 Sep 2012 18:19:25 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id DFF358FC19 for ; Fri, 7 Sep 2012 18:19:24 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 4A703B945; Fri, 7 Sep 2012 14:19:21 -0400 (EDT) From: John Baldwin To: Konstantin Belousov Date: Fri, 7 Sep 2012 14:05:28 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) References: <201209071221.37409.jhb@freebsd.org> <20120907164218.GB33100@deviant.kiev.zoral.com.ua> In-Reply-To: <20120907164218.GB33100@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201209071405.28831.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 07 Sep 2012 14:19:21 -0400 (EDT) Cc: freebsd-current@freebsd.org, Svatopluk Kraus Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 18:19:25 -0000 On Friday, September 07, 2012 12:42:18 pm Konstantin Belousov wrote: > On Fri, Sep 07, 2012 at 12:21:37PM -0400, John Baldwin wrote: > > On Friday, September 07, 2012 12:02:08 pm Konstantin Belousov wrote: > > > On Fri, Sep 07, 2012 at 05:12:37PM +0200, Svatopluk Kraus wrote: > > > > On Tue, Sep 4, 2012 at 6:00 PM, John Baldwin wrote: > > > > > On Tuesday, September 04, 2012 9:00:39 am Konstantin Belousov wrote: > > > > >> 2. I do not see what would prevent malicious local user from mmaping > > > > >> arbitrary file readonly with MAP_TEXT, thus blocking any modifications > > > > >> to the file. Note that this is not a problem for executables, because > > > > >> kernel only sets VV_TEXT on executables if +x permission is set and > > > > >> file is valid binary which kernel is able to execute. > > > > >> > > > > >> E.g. you might block log writes with VV_TEXT, or other user editing > > > > >> session or whatever, having just read access to corresponding files. > > > > >> > > > > >> Am I wrong ? > > > > > > > > > > Hmm, I do think 2) is a bit of a show-stopper. I do wonder why one needs > > > > > MAP_TEXT at all or if you could key this off of mmap() with PROT_EXEC? > > > > > Do we require +x permissions for PROT_EXEC? No, it seems we only require > > > > > a file opened with FREAD. Hmm, perhaps rtld could open a separate fd for > > > > > PROT_EXEC mappings that used O_EXEC and mmap()'ing an O_EXEC fd could enable > > > > > VV_TEXT? That would require a file to have +x permisson for an mmap() to > > > > > enable VV_TEXT. It would also make MAP_TEXT unneeded. > > > > > > > > It sounds good for me. I will try to patch it this way. However, do > > > > you think that will be acceptable to set +x permission to shared > > > > libraries in general? > > > > Shared libraries have +x by default already. You could make rtld fall back > > to using the O_RDONLY fd if it can't open an O_EXEC fd in which case you don't > > get the VV_TEXT protection, but rtld would be backwards compatible. > Where is it ? On fresh stable/9 machine, I have in /lib: > -r--r--r-- 1 root wheel 7216 Dec 3 2011 libipx.so.5 > -r--r--r-- 1 root wheel 19800 Jun 28 18:33 libjail.so.1 > -r--r--r-- 1 root wheel 13752 Jun 28 18:33 libkiconv.so.4 > ... Hmm, I was looking in /usr/local/lib. It seems at least some ports do install libraries with +x: -rwxr-xr-x 1 root wheel 210494 Apr 8 2011 /usr/local/lib/libxml++-2.6.so.2 -rwxr-xr-x 1 root wheel 1515416 Apr 8 2011 /usr/local/lib/libxml2.so.5 -rwxr-xr-x 1 root wheel 44389 Apr 8 2011 /usr/local/lib/libxmlparse.so.1 -rwxr-xr-x 1 root wheel 100672 Apr 8 2011 /usr/local/lib/libxmltok.so.1 -rwxr-xr-x 1 root wheel 266775 Apr 8 2011 /usr/local/lib/libxslt.so.2 -rw-r--r-- 1 root wheel 764602 Apr 8 2011 /usr/local/lib/libxvidcore.so.4 -rwxr-xr-x 1 root wheel 53379 Apr 9 2011 /usr/local/lib/libzip.so.1 > > > Setting +x on shared libraries can be done. But setting VV_TEXT for > > > such mappings is definitely non-standard behaviour, that could cause > > > locking surprises for unaware system administrator. The issuw would be > > > very hard to diagnose. > > > > I'm not sure it would be that obscure. It's the same as getting an error that > > a binary is busy. In that case you would resort to 'ps' to find the offending > > process. For this case (a shared library being busy), you could look at the > > output of 'procstat -af' to find which processes have executable mappings of > > the library. > > I much more worry about rtld reporting 'shared library is busy'. It is fine > to not be able to write to mapped library. > > Rtld errors are usually quite worrying for users, and they just do not > understand them. I think these would be rare? There's no good reason for anything to write to a shared library that I can think of. install(1) does an atomic rename to swap in the new libraries already. -- John Baldwin From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 18:41:26 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EA2901065673; Fri, 7 Sep 2012 18:41:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 6AE7B8FC12; Fri, 7 Sep 2012 18:41:24 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q87IfWFr031493; Fri, 7 Sep 2012 21:41:32 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q87IfKKd072255; Fri, 7 Sep 2012 21:41:20 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q87IfKGd072254; Fri, 7 Sep 2012 21:41:20 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 7 Sep 2012 21:41:20 +0300 From: Konstantin Belousov To: John Baldwin Message-ID: <20120907184120.GD33100@deviant.kiev.zoral.com.ua> References: <201209071221.37409.jhb@freebsd.org> <20120907164218.GB33100@deviant.kiev.zoral.com.ua> <201209071405.28831.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="3vZa96L4PgTIUS1Z" Content-Disposition: inline In-Reply-To: <201209071405.28831.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-current@freebsd.org, Svatopluk Kraus Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 18:41:26 -0000 --3vZa96L4PgTIUS1Z Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Sep 07, 2012 at 02:05:28PM -0400, John Baldwin wrote: > On Friday, September 07, 2012 12:42:18 pm Konstantin Belousov wrote: > > On Fri, Sep 07, 2012 at 12:21:37PM -0400, John Baldwin wrote: > > > On Friday, September 07, 2012 12:02:08 pm Konstantin Belousov wrote: > > > > On Fri, Sep 07, 2012 at 05:12:37PM +0200, Svatopluk Kraus wrote: > > > > > On Tue, Sep 4, 2012 at 6:00 PM, John Baldwin wr= ote: > > > > > > On Tuesday, September 04, 2012 9:00:39 am Konstantin Belousov w= rote: > > > > > >> 2. I do not see what would prevent malicious local user from m= maping > > > > > >> arbitrary file readonly with MAP_TEXT, thus blocking any mo= difications > > > > > >> to the file. Note that this is not a problem for executable= s, because > > > > > >> kernel only sets VV_TEXT on executables if +x permission is= set and > > > > > >> file is valid binary which kernel is able to execute. > > > > > >> > > > > > >> E.g. you might block log writes with VV_TEXT, or other user= editing > > > > > >> session or whatever, having just read access to correspondi= ng files. > > > > > >> > > > > > >> Am I wrong ? > > > > > > > > > > > > Hmm, I do think 2) is a bit of a show-stopper. I do wonder why= one needs > > > > > > MAP_TEXT at all or if you could key this off of mmap() with PRO= T_EXEC? > > > > > > Do we require +x permissions for PROT_EXEC? No, it seems we on= ly require > > > > > > a file opened with FREAD. Hmm, perhaps rtld could open a separ= ate fd for > > > > > > PROT_EXEC mappings that used O_EXEC and mmap()'ing an O_EXEC fd= could enable > > > > > > VV_TEXT? That would require a file to have +x permisson for an= mmap() to > > > > > > enable VV_TEXT. It would also make MAP_TEXT unneeded. > > > > >=20 > > > > > It sounds good for me. I will try to patch it this way. However, = do > > > > > you think that will be acceptable to set +x permission to shared > > > > > libraries in general? > > >=20 > > > Shared libraries have +x by default already. You could make rtld fal= l back > > > to using the O_RDONLY fd if it can't open an O_EXEC fd in which case = you don't > > > get the VV_TEXT protection, but rtld would be backwards compatible. > > Where is it ? On fresh stable/9 machine, I have in /lib: > > -r--r--r-- 1 root wheel 7216 Dec 3 2011 libipx.so.5 > > -r--r--r-- 1 root wheel 19800 Jun 28 18:33 libjail.so.1 > > -r--r--r-- 1 root wheel 13752 Jun 28 18:33 libkiconv.so.4 > > ... >=20 > Hmm, I was looking in /usr/local/lib. It seems at least some ports do > install libraries with +x: >=20 > -rwxr-xr-x 1 root wheel 210494 Apr 8 2011 /usr/local/lib/libxml++-= 2.6.so.2 > -rwxr-xr-x 1 root wheel 1515416 Apr 8 2011 /usr/local/lib/libxml2.s= o.5 > -rwxr-xr-x 1 root wheel 44389 Apr 8 2011 /usr/local/lib/libxmlpar= se.so.1 > -rwxr-xr-x 1 root wheel 100672 Apr 8 2011 /usr/local/lib/libxmltok= .so.1 > -rwxr-xr-x 1 root wheel 266775 Apr 8 2011 /usr/local/lib/libxslt.s= o.2 > -rw-r--r-- 1 root wheel 764602 Apr 8 2011 /usr/local/lib/libxvidco= re.so.4 > -rwxr-xr-x 1 root wheel 53379 Apr 9 2011 /usr/local/lib/libzip.so= .1 >=20 > > > > Setting +x on shared libraries can be done. But setting VV_TEXT for > > > > such mappings is definitely non-standard behaviour, that could cause > > > > locking surprises for unaware system administrator. The issuw would= be > > > > very hard to diagnose. > > >=20 > > > I'm not sure it would be that obscure. It's the same as getting an e= rror that > > > a binary is busy. In that case you would resort to 'ps' to find the = offending > > > process. For this case (a shared library being busy), you could look= at the > > > output of 'procstat -af' to find which processes have executable mapp= ings of > > > the library. > >=20 > > I much more worry about rtld reporting 'shared library is busy'. It is = fine > > to not be able to write to mapped library. > >=20 > > Rtld errors are usually quite worrying for users, and they just do not > > understand them. >=20 > I think these would be rare? There's no good reason for anything to writ= e to > a shared library that I can think of. install(1) does an atomic rename t= o swap > in the new libraries already. After a second thought, I do not like your proposal as well. +x is set for shebang scripts, and allowing PROT_EXEC to set VV_TEXT for them means that such scripts are subject for write denial. --3vZa96L4PgTIUS1Z Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlBKP9AACgkQC3+MBN1Mb4iagQCfWga+wKweqz6Q5mfvr9T2ZJyb eMoAoJQ5t9o0QLDd6mk64KcmXmoRlhy4 =Upo/ -----END PGP SIGNATURE----- --3vZa96L4PgTIUS1Z-- From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 18:43:51 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 145261065674 for ; Fri, 7 Sep 2012 18:43:51 +0000 (UTC) (envelope-from chip@2bithacker.net) Received: from mail.2bithacker.net (unknown [IPv6:2001:470:1f07:202::123]) by mx1.freebsd.org (Postfix) with ESMTP id D3C9A8FC0C for ; Fri, 7 Sep 2012 18:43:50 +0000 (UTC) Received: from 2bithacker.net (nat-04-mht.dyndns.com [216.146.45.243]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: chip) by mail.2bithacker.net (Postfix) with ESMTPSA id 7E268514A1 for ; Fri, 7 Sep 2012 14:43:44 -0400 (EDT) Date: Fri, 7 Sep 2012 14:43:43 -0400 From: Chip Marshall To: freebsd-current@freebsd.org Message-ID: <20120907184342.GB66351@2bithacker.net> Mail-Followup-To: chip@2bithacker.net, freebsd-current@freebsd.org References: <20120830141939.GJ64447@ithaqua.etoilebsd.net> <5048D2EC.70109@FreeBSD.org> <5049E060.9020602@infracaninophile.co.uk> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="8P1HSweYDcXXzwPJ" Content-Disposition: inline In-Reply-To: X-OS: Mac OS X 10.6.8 i386 up 16 days User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: pkg (aka pkgng) 1.0 released X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: chip@2bithacker.net List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 18:43:51 -0000 --8P1HSweYDcXXzwPJ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 07-Sep-2012, Ivan Voras sent: > Ok, but all that can be done the "normal" way with A records. >=20 > As far as I can tell, the intended benefits of the SRV record system > is to disentangle services and hosts, so that, e.g. the same > user-visible DNS name (e.g. "pkg.freebsd.org") resolves to a different > host if asked for the HTTP service and the FTP service. There are some other benefits to using SRV records for this, as it includes port information along with weighting and priority, allowing the servers to run on non-standard ports, and for clients to be able to do some degree of load balancing and failover. There's no easy way to do that with just A records. --=20 Chip Marshall http://weblog.2bithacker.net/ KB1QYW PGP key ID 43C4819E v4sw5PUhw4/5ln5pr5FOPck4ma4u6FLOw5Xm5l5Ui2e4t4/5ARWb7HKOen6a2Xs5IMr2g6CM --8P1HSweYDcXXzwPJ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (Darwin) iEYEARECAAYFAlBKQF4ACgkQnTUxIUPEgZ6JtwCg0ClPUl316LMKyBrrCIcrOJfg BesAoMQW2CDTxqvOgoxDgd7ECZxM7AIF =nrA0 -----END PGP SIGNATURE----- --8P1HSweYDcXXzwPJ-- From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 18:48:30 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CBB7B106566B; Fri, 7 Sep 2012 18:48:30 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 8A24B8FC16; Fri, 7 Sep 2012 18:48:30 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q87ImMGO024839; Fri, 7 Sep 2012 12:48:22 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q87ImJx8045178; Fri, 7 Sep 2012 12:48:19 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Konstantin Belousov In-Reply-To: <20120907184120.GD33100@deviant.kiev.zoral.com.ua> References: <201209071221.37409.jhb@freebsd.org> <20120907164218.GB33100@deviant.kiev.zoral.com.ua> <201209071405.28831.jhb@freebsd.org> <20120907184120.GD33100@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset="us-ascii" Date: Fri, 07 Sep 2012 12:48:19 -0600 Message-ID: <1347043699.1143.2.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, Svatopluk, Kraus Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 18:48:30 -0000 On Fri, 2012-09-07 at 21:41 +0300, Konstantin Belousov wrote: > After a second thought, I do not like your proposal as well. +x is set for > shebang scripts, and allowing PROT_EXEC to set VV_TEXT for them means > that such scripts are subject for write denial. You say that like it's a bad thing. I hate it when I accidentally edit a script that's running and then the script fails because I did. I would be much happier if it acted just like any other executable and prevented modification while it's running. -- Ian From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 18:53:14 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 31E5B106564A; Fri, 7 Sep 2012 18:53:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 9C9D68FC08; Fri, 7 Sep 2012 18:53:13 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q87IrHtp032633; Fri, 7 Sep 2012 21:53:17 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q87Ir5b7072322; Fri, 7 Sep 2012 21:53:05 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q87Ir5pY072321; Fri, 7 Sep 2012 21:53:05 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 7 Sep 2012 21:53:05 +0300 From: Konstantin Belousov To: Ian Lepore Message-ID: <20120907185305.GE33100@deviant.kiev.zoral.com.ua> References: <201209071221.37409.jhb@freebsd.org> <20120907164218.GB33100@deviant.kiev.zoral.com.ua> <201209071405.28831.jhb@freebsd.org> <20120907184120.GD33100@deviant.kiev.zoral.com.ua> <1347043699.1143.2.camel@revolution.hippie.lan> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="p1Ewym4VCenIjxkR" Content-Disposition: inline In-Reply-To: <1347043699.1143.2.camel@revolution.hippie.lan> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-current@freebsd.org, Svatopluk Kraus Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 18:53:14 -0000 --p1Ewym4VCenIjxkR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Sep 07, 2012 at 12:48:19PM -0600, Ian Lepore wrote: > On Fri, 2012-09-07 at 21:41 +0300, Konstantin Belousov wrote: > > After a second thought, I do not like your proposal as well. +x is set = for > > shebang scripts, and allowing PROT_EXEC to set VV_TEXT for them means > > that such scripts are subject for write denial. >=20 > You say that like it's a bad thing. I hate it when I accidentally edit > a script that's running and then the script fails because I did. I > would be much happier if it acted just like any other executable and > prevented modification while it's running. For me, if other user can block my modifications of my script by the mere fact that script has o+rx rights, is indeed bad. I do use real machines sometime. --p1Ewym4VCenIjxkR Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlBKQpEACgkQC3+MBN1Mb4ipQQCfbNRYlkIL48U63ReDclHnuvt/ RgsAn380CgsxA4ED0/OVBA3FruwghmZy =lxhk -----END PGP SIGNATURE----- --p1Ewym4VCenIjxkR-- From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 19:12:18 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 439B2106566B for ; Fri, 7 Sep 2012 19:12:18 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 0224C8FC12 for ; Fri, 7 Sep 2012 19:12:17 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id q87JCGd8025235 for ; Fri, 7 Sep 2012 13:12:17 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id q87JCETM045199; Fri, 7 Sep 2012 13:12:14 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) From: Ian Lepore To: Konstantin Belousov In-Reply-To: <20120907185305.GE33100@deviant.kiev.zoral.com.ua> References: <201209071221.37409.jhb@freebsd.org> <20120907164218.GB33100@deviant.kiev.zoral.com.ua> <201209071405.28831.jhb@freebsd.org> <20120907184120.GD33100@deviant.kiev.zoral.com.ua> <1347043699.1143.2.camel@revolution.hippie.lan> <20120907185305.GE33100@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset="us-ascii" Date: Fri, 07 Sep 2012 13:12:14 -0600 Message-ID: <1347045134.1143.4.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, Svatopluk Kraus Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 19:12:18 -0000 On Fri, 2012-09-07 at 21:53 +0300, Konstantin Belousov wrote: > On Fri, Sep 07, 2012 at 12:48:19PM -0600, Ian Lepore wrote: > > On Fri, 2012-09-07 at 21:41 +0300, Konstantin Belousov wrote: > > > After a second thought, I do not like your proposal as well. +x is set for > > > shebang scripts, and allowing PROT_EXEC to set VV_TEXT for them means > > > that such scripts are subject for write denial. > > > > You say that like it's a bad thing. I hate it when I accidentally edit > > a script that's running and then the script fails because I did. I > > would be much happier if it acted just like any other executable and > > prevented modification while it's running. > > For me, if other user can block my modifications of my script by the mere > fact that script has o+rx rights, is indeed bad. I do use real machines > sometime. But you don't feel the same way about a compiled program? I see absolutely no difference between the two, conceptually. To me, changing an application while it's running is bad. It makes no difference what language the application is written in. -- Ian From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 19:16:37 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 87575106564A for ; Fri, 7 Sep 2012 19:16:37 +0000 (UTC) (envelope-from lokadamus@gmx.de) Received: from mailout-de.gmx.net (mailout-de.gmx.net [213.165.64.23]) by mx1.freebsd.org (Postfix) with SMTP id CBD768FC0C for ; Fri, 7 Sep 2012 19:16:36 +0000 (UTC) Received: (qmail invoked by alias); 07 Sep 2012 19:16:35 -0000 Received: from d165161.adsl.hansenet.de (EHLO FBSD.lokilan) [80.171.165.161] by mail.gmx.net (mp035) with SMTP; 07 Sep 2012 21:16:35 +0200 X-Authenticated: #3333826 X-Provags-ID: V01U2FsdGVkX1++1Y40ThG+qUpjC/gFUMOEnkLGE9GjVV/9pNSVyf bgnV2JnwQBG9Aq Message-ID: <504A4816.4090403@gmx.de> Date: Fri, 07 Sep 2012 21:16:38 +0200 From: "lokadamus@gmx.de" User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:12.0) Gecko/20120505 Thunderbird/12.0.1 MIME-Version: 1.0 To: Ryan Stone References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Cc: FreeBSD Current Subject: Re: machdep.hyperthreading_allowed does not affect SMT cores X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 19:16:37 -0000 On 09/04/12 18:01, Ryan Stone wrote: > I have a Intel Sandy Bridge system that reports that it has SMT cores > instead of HTT(under a derivative of FreeBSD 8.2). I'll admit that I > don't at all understand the distinction between the two -- I thought > that HTT was just Intel's name for SMT. In any case, is there any > reason that machdep.hyperthreading_allowed should not apply to SMT > cores, too? > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > No, HTT is a half core. It will speed up a system round about 20%, but only when it can work with HTT. HTT is the idea to work on the last thing while new data will transfer from ram to cpu cache. Else the cpu wlll idle and doing nothing. Problem is to use htt on a heavy loaded system. Since Win7 Microsoft use it on a good way. Bevor this, they make a mistake to use it as a normale core. Under Vista cpu 0 is under heavy load they put it to cpu 1, but this was only a htt core. On a dualcore core 2 must use, bevor core 1 (htt) is used. This it is, what MS change. From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 19:41:25 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5F3041065673 for ; Fri, 7 Sep 2012 19:41:25 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 3363E8FC0A for ; Fri, 7 Sep 2012 19:41:25 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 96244B93B; Fri, 7 Sep 2012 15:41:24 -0400 (EDT) From: John Baldwin To: Konstantin Belousov Date: Fri, 7 Sep 2012 15:40:42 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) References: <201209071405.28831.jhb@freebsd.org> <20120907184120.GD33100@deviant.kiev.zoral.com.ua> In-Reply-To: <20120907184120.GD33100@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201209071540.43013.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 07 Sep 2012 15:41:24 -0400 (EDT) Cc: freebsd-current@freebsd.org, Svatopluk Kraus Subject: Re: [patch] mmap() MAP_TEXT implementation (to use for shared libraries) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 19:41:25 -0000 On Friday, September 07, 2012 2:41:20 pm Konstantin Belousov wrote: > > I think these would be rare? There's no good reason for anything to write to > > a shared library that I can think of. install(1) does an atomic rename to swap > > in the new libraries already. > > After a second thought, I do not like your proposal as well. +x is set for > shebang scripts, and allowing PROT_EXEC to set VV_TEXT for them means > that such scripts are subject for write denial. Yeah, that's fair. Also, I hunted around to find the description of MAP_TEXT in Solaris 11. It seems from reading that that MAP_TEXT on Solaris isn't used to prevent writes ala VV_TEXT. Instead, it is used as a hint that is apparently used to use superpages for text. -- John Baldwin From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 19:51:26 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0D646106566C for ; Fri, 7 Sep 2012 19:51:26 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from mail.soaustin.net (pancho.soaustin.net [76.74.250.40]) by mx1.freebsd.org (Postfix) with ESMTP id DE1158FC12 for ; Fri, 7 Sep 2012 19:51:25 +0000 (UTC) Received: by mail.soaustin.net (Postfix, from userid 502) id 766F25623C; Fri, 7 Sep 2012 14:51:25 -0500 (CDT) Date: Fri, 7 Sep 2012 14:51:25 -0500 From: Mark Linimon To: Benjamin Kaduk Message-ID: <20120907195125.GB13779@lonesome.com> References: <5049B1C2.6030604@zedat.fu-berlin.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Eitan Adler , Current FreeBSD , "O. Hartmann" Subject: Re: make: don't know how to make add-plist-buildinfo. Stop X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 19:51:26 -0000 It was a temporary disruption that I caused. It should be fixed now. mcl From owner-freebsd-current@FreeBSD.ORG Fri Sep 7 21:23:33 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A34061065672; Fri, 7 Sep 2012 21:23:33 +0000 (UTC) (envelope-from simon@FreeBSD.org) Received: from emx.nitro.dk (emx.nitro.dk [IPv6:2a01:4f8:120:7384::102]) by mx1.freebsd.org (Postfix) with ESMTP id 235988FC08; Fri, 7 Sep 2012 21:23:33 +0000 (UTC) Received: from mailscan.leto.nitro.dk (mailscan.leto.nitro.dk [127.0.1.4]) by emx.nitro.dk (Postfix) with ESMTP id 42F592CBE67; Fri, 7 Sep 2012 21:23:32 +0000 (UTC) Received: from emx.nitro.dk ([127.0.1.2]) by mailscan.leto.nitro.dk (mailscan.leto.nitro.dk [127.0.1.4]) (amavisd-new, port 10024) with LMTP id 7Ga3AHdYUcvg; Fri, 7 Sep 2012 21:23:30 +0000 (UTC) Received: from [192.168.4.35] (unknown [89.100.2.68]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by emx.nitro.dk (Postfix) with ESMTPSA id 03FBA2CBE61; Fri, 7 Sep 2012 21:23:29 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) From: "Simon L. B. Nielsen" In-Reply-To: Date: Fri, 7 Sep 2012 22:23:30 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20120830141939.GJ64447@ithaqua.etoilebsd.net> <5048D2EC.70109@FreeBSD.org> <5049E060.9020602@infracaninophile.co.uk> To: Ivan Voras X-Mailer: Apple Mail (2.1486) Cc: Matthew Seaman , freebsd-current@freebsd.org Subject: Re: pkg (aka pkgng) 1.0 released X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Sep 2012 21:23:33 -0000 On 7 Sep 2012, at 13:02, Ivan Voras wrote: > On 7 September 2012 13:54, Matthew Seaman > wrote: >> On 09/07/12 12:30, Ivan Voras wrote: >>> On 06/09/2012 18:44, Matthew Seaman wrote: >>>> On 06/09/2012 16:37, Ivan Voras wrote: >>>=20 >>>>> Hi, >>>>>=20 >>>>> It looks like the pkg port installs pkg.conf.sample with the line: >>>>>=20 >>>>> PACKAGESITE : http://pkg.freebsd.org/${ABI}/latest >>>>>=20 >>>>> ... which is finally a good step in the direction of making pkgng >>>>> working by default, except that the "pkg.freebsd.org" site doesn't = exist >>>>> in DNS. Instead, pkgbeta.freebsd.org still exists. I suppose one = should >>>>> be a DNS CNAME for the other? >>>>=20 >>>> It's a SRV record: >>>>=20 >>>> seedling:~:% dig +short IN SRV _http._tcp.pkg.freebsd.org >>>> 10 10 80 pkgbeta.FreeBSD.org. >>>=20 >>> Hi, >>>=20 >>> What are the benefits of doing it this way? >>=20 >> Yeah -- it's a bit OTT right now given there's just the one publicly >> available pkg repository available. It will pay off later when there >> are more pkg repositories available -- repositories can be added to = (or >> removed from) the list in the SRV record without end-users having to >> know the details. >>=20 >> It may also be possible to replicate what portupdate has done and use >> geolocation based services to steer end users to a nearby repository >> site automatically. >=20 > Ok, but all that can be done the "normal" way with A records. >=20 > As far as I can tell, the intended benefits of the SRV record system > is to disentangle services and hosts, so that, e.g. the same > user-visible DNS name (e.g. "pkg.freebsd.org") resolves to a different > host if asked for the HTTP service and the FTP service. That is one feature of SRV, but that part isn't really something we use = for anything, nor plan to (other than the fact that you specify SRV = records that way). > I suppose this could work in FreeBSD's case if the record was created > differently, instad of the "normal" _http._tcp record, introduce a new > one, e.g. _pkgng._tcp, so when a web browser visits "pkg.freebsd.org" > it gets a regular web page or some other user-visible content, and the There is no plan to add an A record to pkg.freebsd.org, so a browser = would never be able to resolve it. Adding an A record there would IMO be = more likely to mislead people to think that pkg(8) (is it section 8?) = fetches packages from there. > specially made pkg client will resolve it to another service and > another (possibly) server. This also can be done without the SRV > record, by e.g. inspecting client's HTTP headers. >=20 > I'm not saying that the SRV record is technically wrong in this case, > I just don't see how is it useful (and surely there will be others > trying to ping "pkg.freebsd.org" and complaining when it fails to > resolve). As Chip Marshall mentions, SRV records allow us to load balancing and = failover. Both are rather important in allowing us to scale pkg = distribution and changing it without requiring client side changes. The = fallback handling of SRV allows us, among other things, to have thin = frontend servers which only have a subset of packages, and then = automatic fallback to the full mirrors. Once the initial pkg distribution sites are set up, we will also create = some more SRV records, so the people who want can prioritize mirrors = close to them based on regions while still falling back to mirrors = longer away from them. Oh, and anyone using portsnap or freebsd-update are already using SRV = records. For anyone interested in more details, they can read my document = describing the system, which bapt implemented the SRV support in pkgng = from: = https://docs.google.com/document/d/1MmQOV2IUfRtdlzd1i63SJQgngjPJ8eERaSa68X= ydyc0/edit --=20 Simon L. B. Nielsen From owner-freebsd-current@FreeBSD.ORG Sat Sep 8 08:35:15 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6E3C106564A; Sat, 8 Sep 2012 08:35:15 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id 8679F8FC0C; Sat, 8 Sep 2012 08:35:15 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.69) with esmtp (envelope-from ) id <1TAGVW-0002qf-FU>; Sat, 08 Sep 2012 10:35:14 +0200 Received: from e178019143.adsl.alicedsl.de ([85.178.19.143] helo=thor.walstatt.dyndns.org) by inpost2.zedat.fu-berlin.de (Exim 4.69) with esmtpsa (envelope-from ) id <1TAGVW-0003n4-9b>; Sat, 08 Sep 2012 10:35:14 +0200 Message-ID: <504B033B.4020001@zedat.fu-berlin.de> Date: Sat, 08 Sep 2012 10:35:07 +0200 From: "O. Hartmann" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120810 Thunderbird/14.0 MIME-Version: 1.0 To: Brooks Davis References: <5049C13E.5010808@mail.zedat.fu-berlin.de> <504A0E46.3010306@FreeBSD.org> <504A16BA.7030407@mail.zedat.fu-berlin.de> <20120907170728.GB28113@lor.one-eyed-alien.net> In-Reply-To: <20120907170728.GB28113@lor.one-eyed-alien.net> X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig98553C4A03C9C0FA0F70B63A" X-Originating-IP: 85.178.19.143 Cc: "O. Hartmann" , Current FreeBSD , Dimitry Andric Subject: Re: FreeBSD 10.0-CURRENT: CLANG and port/clang weirdness! X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 08:35:16 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig98553C4A03C9C0FA0F70B63A Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 09/07/12 19:07, Brooks Davis wrote: > On Fri, Sep 07, 2012 at 05:46:02PM +0200, O. Hartmann wrote: >> On 09/07/12 17:09, Dimitry Andric wrote: >>> On 2012-09-07 11:41, O. Hartmann wrote: >>>> Building ports not explicitely enabling USE_GCC=3D4.6+ are considere= d >>>> using the system's LLVM/CLANG, which is clang 3.2 in our installatio= n >>>> (FreeBSD 10.0-CURRENT #0 r240164), but since some ports require the >>>> special ports devel/llvm and lang/clang, LLVM 3.1 and clang 3.1 get >>>> installed and 3.1 is used instead the system's 3.2 whenever "clang",= >>>> "clang++" is invoked. >>> >>> Maybe a solution would be to use the same approach as with the gcc >>> ports, namely installing the clang 3.1 executables into /usr/local/bi= n >>> as clang-3.1, clang++-3.1 and clang-cpp-3.1. Then you could simply s= et >>> >>> CC=3Dclang-3.1 >>> CXX=3Dclang++-3.1 >>> CPP=3Dclang-cpp-3.1 >>> >>> for the targets that require it. Brooks? :) >> >> I would appreciate such an approach, since it would be consistent with= >> with GCC, as you stated. >=20 > I'd like to do this, but it doesn't look like it will be easy. There > appears to be no support in the llvm build system for it. :( >=20 >>>> Following the WIKI at http://wiki.freebsd.org/BuildingFreeBSDWithCla= ng >>>> introduces the usage of >>>> >>>> CC=3Dclang instead of CC=3D/usr/bin/clang >>>> CXX=3Dclang++ instead of CXX=3D/usr/bin/clang++ >>>> CPP=3Dclang-ccp instead of CPP=3D/usr/bin/clang-ccp >>>> >>>> Is this intended? >>> >>> Yes. During buildworld, in the cross-tools stage, a new compiler is >>> built, and it is placed under ${WORLDTMP}, usually /usr/obj/usr/src/t= mp. >>> Afterwards, in the rest of the stages, the PATH is changed so >>> executables from ${WORLDTMP} are preferred above those in the system >>> directories. >>> >>> Therefore, if you set CC/CXX/CPP with an explicit path, this logic wi= ll >>> not work, and your buildworld may have all kinds of trouble. I think= >>> there are several patches floating around to fix this, in various >>> different ways. >> >> Understood. >=20 > FWIW, picking up clang etc from /usr/local should be mostly harmless > during the early build stage. You're actual world will be built with > the cross clang. =2E.. means, the resulting WORLD and KERNEL is then build by the LLVM/CLANG that is residing in /usr/obj/...? But what is with PORTS I build later? They definitely pick up the "wrong" clang/clang++. I was suggested to deinstall (or not to install) the port's version of LLVM/CLANG, but this happens automatically for some ports - like LibreOffice. >=20 >> Since I'm using on most boxes 10.0, where can I read more about the ne= w >> toolchain approach? >=20 > Discussions will take place on the toolchain@ list. I'll be posting > some writeups soon. >=20 >> And by the way - is there a way to have also LLVM installed with the >> base system by a knob like "WITH_LLVM"? This would avoid the same >> confusion as mentioned above with CLANG when it comes to LLVM >> dependencies (I play around with some OpenCL libs (pocl), which seems = to >> need LLVM port installed). >=20 > You're probably looking for WITH_CLANG_EXTRAS. I already have that option enabled in my /etc/src.conf. But unfortunately, a tool called "llvm-config" and sibblings weren't installed, they get installed only with /usr/ports/devel/llvm[-devel]. I discovered this playing around with LLVM and there is a software package I play with (pocl, portable OpenCL library) which uses LLVM backe= nd. At this point, my "point of view" might me wrong, but I look at CLANG as it is a part of the LLVM framework and so I inherently expected it to be existent (the LLVM) in FreeBSD. But I guess the policy in FBSD is to have only the clang and the necessary portions of LLVM in the tree. So I thought it would be nice to have the "magic knob" in case one will take the burden of compiling more and time consuming more LLVM/CLANG stuff if he chooses to do. >=20 > -- Brooks >=20 >> >> Thanks for your response, >> >> regards >> >> Oliver >> >> >=20 --------------enig98553C4A03C9C0FA0F70B63A Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBAgAGBQJQSwNBAAoJEOgBcD7A/5N8oSUIAMjHHoaXaxlFQaqdjqQ8yDFp BnbpW+fcy0OfRtGtIZQ3r9it9rGtBI0leqtbQLQ/v1nM+wyRDSUwMo37LKR2Jo1A TPqpe4lkXBw8gF1TuEQXjU6FUc1l24x7gAdYcsvuJMFd4i2vvm9iWuib1TuJ2RRD 25v1SDfqTrfnNb5TZMan7P/AdYgBYchg2X0N7kIUAcpn6rrTJ8YhlHpx7pJIczGG VkECwXrna9cCYSCTFn3RpadxjOuCTEJFXF7SDMl1WI1R7R+5+9FiB9lSh1EpNhS1 GqcvsTfYuwFuicBPZVwMAQ12etVLDjmSlEVb1wmZdFbf0OsCQdi6HT0l2neNPLE= =hZsG -----END PGP SIGNATURE----- --------------enig98553C4A03C9C0FA0F70B63A-- From owner-freebsd-current@FreeBSD.ORG Sat Sep 8 08:57:56 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A65A106564A; Sat, 8 Sep 2012 08:57:56 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.freebsd.org (Postfix) with ESMTP id 0CE858FC08; Sat, 8 Sep 2012 08:57:55 +0000 (UTC) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.69) with esmtp (envelope-from ) id <1TAGrT-0004li-5u>; Sat, 08 Sep 2012 10:57:55 +0200 Received: from e178019143.adsl.alicedsl.de ([85.178.19.143] helo=thor.walstatt.dyndns.org) by inpost2.zedat.fu-berlin.de (Exim 4.69) with esmtpsa (envelope-from ) id <1TAGrS-0004pf-Vg>; Sat, 08 Sep 2012 10:57:55 +0200 Message-ID: <504B088D.2030709@zedat.fu-berlin.de> Date: Sat, 08 Sep 2012 10:57:49 +0200 From: "O. Hartmann" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120810 Thunderbird/14.0 MIME-Version: 1.0 To: Current FreeBSD , Ports FreeBSD X-Enigmail-Version: 1.4.3 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigCA7D68C5F38E1C801AEF73C4" X-Originating-IP: 85.178.19.143 Cc: Subject: HELP: New Port: devel/pocl: A portable OpenCL library with LLVM backend X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 08:57:56 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigCA7D68C5F38E1C801AEF73C4 Content-Type: multipart/mixed; boundary="------------020103060009000209060501" This is a multi-part message in MIME format. --------------020103060009000209060501 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello. I tried to create a "port", see the Makefile attached I created already. For further informations and your convenience, look at this website: POCL: https://launchpad.net/pocl LLVM: http://llvm.org/docs/ReleaseNotes.html I use FreeBSD 10.0-CURRENT r240186M amd64, my world and kernel are build with CLANG. The software, pocl, itself is straight forward able to compile. I didn't "patch" anything or put magic sophisticated knowledge into "porting it", I simply created the Makefile for the start. Feel free to work on it. The Makefile is capable of building POCL on all FreeBSD 10 systems I run, but it fails on FreeBSD 9.1-STABLE. I didn't investigate further, since I have some first-priority questiosn and maybe some of the more experienced people here are willing to contribute their knowledge. 0) In my CLANG built world, fetch(1) ends up with an "Authentication error" fetching the sources from the site (see PR: bin/171402). This needs to be fixed before the port works out of the box, you need to download the source tar-ball manually for the first time from the site mentioned above entitle POCL. 1) The port need to be build with a version number of the shared library created - otherwise ldconfig doesn't seem to recognize the installed library and does not load it into the cache. 2) A more sophisticated detection of the build environment is needed. I failed providing a detection of the existence of a CLANG, legacy or GCC > 4.6 build environment, I guess this can be done with the version number of FreeBSD itself or with some knobs I do not know about. I'm new to the ports system on this sophisticated level. I think it is too early to announce this port with a PR, since FreeBSD < 10 isn't supported yet. 3) I didn't test the library with "real world" software, but I think I will this in the near future, if it is working and usable. If some folks could also provide "tests", please be invited. Thanks. Oliver --------------020103060009000209060501 Content-Type: text/plain; charset=us-ascii; name="Makefile" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="Makefile" PORTNAME=3D pocl PORTVERSION=3D 0.6.0 CATEGORIES=3D devel MAINTAINER=3D ports@FreeBSD.org COMMENT=3D Portable OpenCL - an efficient open source (MIT-licensed) imp= lementation of the OpenCL 1.2 standard. MASTER_SITES=3D https://launchpad.net/pocl/${PORTVERSION:C/\.[0-9]$//}/$= {PORTVERSION}/+download/ DISTNAME=3D ${PORTNAME}-${PORTVERSION:C/\.[0-9]$//} CONFLICTS=3D opencl-* freeocl* DISABLE_MAKE_JOBS=3D yes OPTIONS_SINGLE=3D LANG OPTIONS_SINGLE_LANG=3D COMP_CLANG COMP_GCC OPTIONS_DEFAULT=3D COMP_CLANG COMP_CLANG_DESC=3D Use LLVM/CLANG compiler COMP_GCC_DESC=3D Use GNU GCC compiler =2Einclude =2Eif ${PORT_OPTIONS:MCOMP_GCC} USE_GCC=3D 4.6+ =2Eendif =2Eif ${PORT_OPTIONS:MCOMP_CLANG} CXXFLAGS+=3D -stdlib=3Dlibc++ CPPFLAGS+=3D -I${LOCALBASE}/include =2Eendif BUILD_DEPENDS=3D ${LOCALBASE}/bin/llvm-config:${PORTSDIR}/devel/llvm USE_GMAKE=3D yes USE_GL=3D yes USE_LDCONFIG=3D yes USE_AUTOTOOLS=3D libltdl GNU_CONFIGURE=3D yes CPPFLAGS+=3D -I${LOCALBASE}/include LDFLAGS+=3D -L${LOCALBASE}/lib =2Einclude --------------020103060009000209060501-- --------------enigCA7D68C5F38E1C801AEF73C4 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQEcBAEBAgAGBQJQSwiSAAoJEOgBcD7A/5N8XSUH/R+EDEBEuv2yYt7TbbZkNpbk iWHs7AI1fwB6KmuBkzSS9dg+NAmgtWU6Q0QLexyGh4g66zK13gg6dQ9lCvTUnJ5R BXNBd4rfrxLshNKxHt8hG/mIleRq6G2Sz7XJvFMejMkeAbRmR9jOg+3IV+2YjV9H QpOOwIocSRSEQgJggtcqsPOX3hsl2yTt+8bRSDVnMGlkwA/KzWn4mKvapZPaaip7 FNBM0AL6q3UN87OFQhQ9MzCG7B4tGULdO6t6LLoGds4PQG00y37+2U5Zk3si922n xYNtqV8AeG/QNnnxFD7+2MPvqaHTuGiXyvxT99tsB4BjTyYiMPrWP3f5095twWo= =Vgbm -----END PGP SIGNATURE----- --------------enigCA7D68C5F38E1C801AEF73C4-- From owner-freebsd-current@FreeBSD.ORG Sat Sep 8 10:59:05 2012 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AC62C106566B; Sat, 8 Sep 2012 10:59:05 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id 63C298FC12; Sat, 8 Sep 2012 10:59:05 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:7556:32b3:4fd3:596e] (unknown [IPv6:2001:7b8:3a7:0:7556:32b3:4fd3:596e]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id C98E85C37; Sat, 8 Sep 2012 12:59:03 +0200 (CEST) Message-ID: <504B24FF.5070901@FreeBSD.org> Date: Sat, 08 Sep 2012 12:59:11 +0200 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20120905 Thunderbird/16.0 MIME-Version: 1.0 To: "O. Hartmann" References: <5049C13E.5010808@mail.zedat.fu-berlin.de> <504A0E46.3010306@FreeBSD.org> <504A16BA.7030407@mail.zedat.fu-berlin.de> <20120907170728.GB28113@lor.one-eyed-alien.net> <504B033B.4020001@zedat.fu-berlin.de> In-Reply-To: <504B033B.4020001@zedat.fu-berlin.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Current FreeBSD , "O. Hartmann" , Brooks Davis Subject: Re: FreeBSD 10.0-CURRENT: CLANG and port/clang weirdness! X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 10:59:05 -0000 On 2012-09-08 10:35, O. Hartmann wrote: > On 09/07/12 19:07, Brooks Davis wrote: ... >> FWIW, picking up clang etc from /usr/local should be mostly harmless >> during the early build stage. You're actual world will be built with >> the cross clang. > ... means, the resulting WORLD and KERNEL is then build by the > LLVM/CLANG that is residing in /usr/obj/...? Yes. This is done so you can build an updated version of the system compiler during the cross-tools stage, and then build the rest of the world with this updated compiler. > But what is with PORTS I build later? They definitely pick up the > "wrong" clang/clang++. If your search path has /usr/local/bin before /usr/bin, it will obviously always pick up the ports-compiled clang, whatever version that is. If some other ports explicitly require clang 3.1, there should be a mechanism to specifically invoke that version, instead of whatever the first "clang" executable in the search path is. For now, I think the best solution is to use the following (admittedly rather kludgy) fragment in /etc/make.conf: .if ${.CURDIR:M/usr/src*} || ${.CURDIR:M/usr/obj*} CC=clang CXX=clang++ CPP=clang-cpp .else CC=/usr/bin/clang CXX=/usr/bin/clang++ CPP=/usr/bin/clang-cpp .endif Or alternatively: .if ${.CURDIR:M/usr/ports*} CC=/usr/bin/clang CXX=/usr/bin/clang++ CPP=/usr/bin/clang-cpp .else CC=clang CXX=clang++ CPP=clang-cpp .endif >> You're probably looking for WITH_CLANG_EXTRAS. > I already have that option enabled in my /etc/src.conf. But > unfortunately, a tool called "llvm-config" and sibblings weren't > installed, they get installed only with /usr/ports/devel/llvm[-devel]. This is on purpose, at least for now. The base system should only contain the clang compiler proper, and none of the LLVM libraries. The WITH_CLANG_EXTRAS option is just for getting a few tools that could be handy, but should not be used by 99% of the users out there. If you want the whole llvm and/or clang shebang, including the shared libraries, just install the port. You can use the lang/clang-devel port if you want to try out the bleeding edge version. AFAIK Brooks updates it regularly. From owner-freebsd-current@FreeBSD.ORG Sat Sep 8 12:15:20 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 512581065670 for ; Sat, 8 Sep 2012 12:15:20 +0000 (UTC) (envelope-from se@freebsd.org) Received: from nm11-vm0.bullet.mail.ukl.yahoo.com (nm11-vm0.bullet.mail.ukl.yahoo.com [217.146.183.244]) by mx1.freebsd.org (Postfix) with SMTP id BBAA48FC0C for ; Sat, 8 Sep 2012 12:15:19 +0000 (UTC) Received: from [217.146.183.209] by nm11.bullet.mail.ukl.yahoo.com with NNFMP; 08 Sep 2012 12:15:13 -0000 Received: from [77.238.184.58] by tm2.bullet.mail.ukl.yahoo.com with NNFMP; 08 Sep 2012 12:15:13 -0000 Received: from [127.0.0.1] by smtp127.mail.ukl.yahoo.com with NNFMP; 08 Sep 2012 12:15:13 -0000 X-Yahoo-Newman-Id: 662641.75710.bm@smtp127.mail.ukl.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: QGG68QwVM1mziHVcE.Cz55GVlv6je8feJh8vhNpGg.GdPYo fnhslXUlMbk.a7DW5g_Sjme_WIqQZygUAOwylpHwePI_gKOE4nDHb.rarpq7 uWh2kqecYu39WWw1hOj8oY18Qx6sf2Khruh67NL2o5umWyE0Gfi7rjiLGm.I cwEIqb_lg0YgtOq_H1zRacNb.3qX5sMUW2uFXglh.9OA6.zey8LaQqbXBk8D fpBI1bF5oiXChr13MYcoisiCYltxBAnie7by2MsqPNJkiazbQApPw4.iLyNZ cX.Yg.XAE9UA2DH7P7sabWqnsNEVmGYBqQOhv.ag5XFOCzTfuLDxrjV4zzUJ jcHdJLBU7AjEYrJ2xhzRaZuNPU79n_xMc3VrbardX4_2jFR6p4lQeeMO7Cxk LjhcfJx16cm0UUOaF3.gJ8CdKz4_dArgb0tnSKTaI.3ujkz0F88Sw0eEdkrr 4C9E4I838hX0g9goyr1n4NTo75HWqZoFIgWKDcGSPdUvGnlKb0gA- X-Yahoo-SMTP: iDf2N9.swBDAhYEh7VHfpgq0lnq. Received: from [192.168.119.19] (se@81.173.156.71 with plain) by smtp127.mail.ukl.yahoo.com with SMTP; 08 Sep 2012 12:15:13 +0000 GMT Message-ID: <504B36CD.9050905@freebsd.org> Date: Sat, 08 Sep 2012 14:15:09 +0200 From: Stefan Esser User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 To: ohartman@zedat.fu-berlin.de References: <5049C13E.5010808@mail.zedat.fu-berlin.de> <504A0E46.3010306@FreeBSD.org> <504A16BA.7030407@mail.zedat.fu-berlin.de> <20120907170728.GB28113@lor.one-eyed-alien.net> <504B033B.4020001@zedat.fu-berlin.de> In-Reply-To: <504B033B.4020001@zedat.fu-berlin.de> X-Enigmail-Version: 1.4.4 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org Subject: Re: FreeBSD 10.0-CURRENT: CLANG and port/clang weirdness! X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 12:15:20 -0000 Am 08.09.2012 10:35, schrieb O. Hartmann: > On 09/07/12 19:07, Brooks Davis wrote: >> On Fri, Sep 07, 2012 at 05:46:02PM +0200, O. Hartmann wrote: >>> On 09/07/12 17:09, Dimitry Andric wrote: >>>> On 2012-09-07 11:41, O. Hartmann wrote: >>>>> Building ports not explicitely enabling USE_GCC=4.6+ are >>>>> considered using the system's LLVM/CLANG, which is clang >>>>> 3.2 in our installation (FreeBSD 10.0-CURRENT #0 r240164), >>>>> but since some ports require the special ports devel/llvm >>>>> and lang/clang, LLVM 3.1 and clang 3.1 get installed and >>>>> 3.1 is used instead the system's 3.2 whenever "clang", >>>>> "clang++" is invoked. >>>> >>>> Maybe a solution would be to use the same approach as with >>>> the gcc ports, namely installing the clang 3.1 executables >>>> into /usr/local/bin as clang-3.1, clang++-3.1 and >>>> clang-cpp-3.1. Then you could simply set >>>> >>>> CC=clang-3.1 CXX=clang++-3.1 CPP=clang-cpp-3.1 >>>> >>>> for the targets that require it. Brooks? :) >>> >>> I would appreciate such an approach, since it would be >>> consistent with with GCC, as you stated. >> >> I'd like to do this, but it doesn't look like it will be easy. >> There appears to be no support in the llvm build system for it. >> :( >> >>>>> Following the WIKI at >>>>> http://wiki.freebsd.org/BuildingFreeBSDWithClang introduces >>>>> the usage of >>>>> >>>>> CC=clang instead of CC=/usr/bin/clang CXX=clang++ instead >>>>> of CXX=/usr/bin/clang++ CPP=clang-ccp instead of >>>>> CPP=/usr/bin/clang-ccp >>>>> >>>>> Is this intended? >>>> >>>> Yes. During buildworld, in the cross-tools stage, a new >>>> compiler is built, and it is placed under ${WORLDTMP}, >>>> usually /usr/obj/usr/src/tmp. Afterwards, in the rest of the >>>> stages, the PATH is changed so executables from ${WORLDTMP} >>>> are preferred above those in the system directories. >>>> >>>> Therefore, if you set CC/CXX/CPP with an explicit path, this >>>> logic will not work, and your buildworld may have all kinds >>>> of trouble. I think there are several patches floating >>>> around to fix this, in various different ways. >>> >>> Understood. >> >> FWIW, picking up clang etc from /usr/local should be mostly >> harmless during the early build stage. You're actual world will >> be built with the cross clang. > > > ... means, the resulting WORLD and KERNEL is then build by the > LLVM/CLANG that is residing in /usr/obj/...? > > But what is with PORTS I build later? They definitely pick up the > "wrong" clang/clang++. > > I was suggested to deinstall (or not to install) the port's version > of LLVM/CLANG, but this happens automatically for some ports - > like LibreOffice. Simple solution: If /usr/local/bin must be in front of /usr/bin and the port version of CLANG is thus found in preference of the system version, you may still put another directory with symbolic links to system binaries in front of /usr/local/bin. E.g.: PATH=/usr/pref_bin:/usr/local/bin:/usr/bin Directory /usr/pref_bin contains symbolic links to binaries in /usr/bin, e.g. /usr/pref_bin/clang -> /usr/bin/clang. Regards, STefan From owner-freebsd-current@FreeBSD.ORG Sat Sep 8 15:31:22 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7E00106566C for ; Sat, 8 Sep 2012 15:31:22 +0000 (UTC) (envelope-from w8hdkim@gmail.com) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 61AF38FC18 for ; Sat, 8 Sep 2012 15:31:22 +0000 (UTC) Received: by vbmv11 with SMTP id v11so1103969vbm.13 for ; Sat, 08 Sep 2012 08:31:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=b6JUoahKvcVEynxRnieHZb2k8z05k5j6EjUKUB72BZo=; b=lyQwXSsx72WCxhiz8wlv4eQwBCZVEt7uZ/TnhlEFScdagkY42MgBxKP+dHK7go8V59 BWYY2BfkvzdGf4U1y6QQ/pyYmx48HxybTYy3kMxpXR0fU8jt24xHNylDwVEhTRjWHq+Q Pe6npwpn1K/rsmowYyoen0ShiPQ9Lt3aAWkcngty05CVdxbr03JK4doaQoD0/u24+MFo PYgKDsud4hz9MBDvoLTgwPrYoGR8Jg8D2poa3Nuz6szZGZy2WyIJ0NZYTcMsc2AsUVC5 GNabmRtd4faaV4LXu1LDqkMF/leXJRJ+ol8LBiJid3cRC7s7R+izKiGe9M1gmBqEDGSL ATCA== MIME-Version: 1.0 Received: by 10.52.28.113 with SMTP id a17mr9340476vdh.47.1347118281786; Sat, 08 Sep 2012 08:31:21 -0700 (PDT) Received: by 10.58.26.129 with HTTP; Sat, 8 Sep 2012 08:31:21 -0700 (PDT) Date: Sat, 8 Sep 2012 11:31:21 -0400 Message-ID: From: Kim Culhan To: freebsd-current@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: 10-CURRENT clock_getcpuclockid() usage X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 15:31:22 -0000 clock_getcpuclockid() was added a few weeks ago according to the man page and I'm seeing this error while building the port net/freeswith-core-devel. The function is used in the Sofia-sip stack, courtesy Nokia Research Center, incorporated into FreeSWITCH. The build system tests for the presence of clock_getcpuclockid() and finds it but in the error messages 'clock_getcpuclockid2' is mentioned. Any thoughts on the clock_getcpuclockid() usage would be very greatly appreciated. LTCOMPILE su_time.lo su_time.c:410:14: error: implicit declaration of function 'clock_getcpuclockid' is invalid in C99 [-Werror,-Wimplicit-function-declaration] else if (clock_getcpuclockid(0, &cpu) != -1 && ^ su_time.c:410:14: note: did you mean 'clock_getcpuclockid2'? else if (clock_getcpuclockid(0, &cpu) != -1 && ^~~~~~~~~~~~~~~~~~~ clock_getcpuclockid2 /usr/include/sys/time.h:352:5: note: 'clock_getcpuclockid2' declared here int clock_getcpuclockid2(id_t, int, clockid_t *); ^ 1 error generated. gmake[9]: *** [su_time.lo] Error 1 thanks -kim From owner-freebsd-current@FreeBSD.ORG Sat Sep 8 15:35:58 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F08D106566B; Sat, 8 Sep 2012 15:35:58 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) by mx1.freebsd.org (Postfix) with ESMTP id C4B498FC0C; Sat, 8 Sep 2012 15:35:57 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.5/8.14.5) with ESMTP id q88F7d7v038989; Sat, 8 Sep 2012 08:07:39 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.5/8.14.5/Submit) id q88F7c0V038988; Sat, 8 Sep 2012 08:07:38 -0700 (PDT) (envelope-from sgk) Date: Sat, 8 Sep 2012 08:07:38 -0700 From: Steve Kargl To: "O. Hartmann" Message-ID: <20120908150738.GA38981@troutmask.apl.washington.edu> References: <504B088D.2030709@zedat.fu-berlin.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <504B088D.2030709@zedat.fu-berlin.de> User-Agent: Mutt/1.4.2.3i Cc: Current FreeBSD , Ports FreeBSD Subject: Re: HELP: New Port: devel/pocl: A portable OpenCL library with LLVM backend X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 15:35:58 -0000 On Sat, Sep 08, 2012 at 10:57:49AM +0200, O. Hartmann wrote: > Hello. > I tried to create a "port", see the Makefile attached I created already. > For further informations and your convenience, look at this website: > Please stop cross posting to freebsd-current when you have some issue that clearly belongs in free-ports. -- Steve From owner-freebsd-current@FreeBSD.ORG Sat Sep 8 16:46:07 2012 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 414D610657CD for ; Sat, 8 Sep 2012 16:46:07 +0000 (UTC) (envelope-from jakub_lach@mailplus.pl) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) by mx1.freebsd.org (Postfix) with ESMTP id 1564A8FC0C for ; Sat, 8 Sep 2012 16:46:06 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1TAOAS-0003ju-FJ for freebsd-current@freebsd.org; Sat, 08 Sep 2012 09:46:00 -0700 Date: Sat, 8 Sep 2012 09:46:00 -0700 (PDT) From: Jakub Lach To: freebsd-current@freebsd.org Message-ID: <1347122760466-5741777.post@n5.nabble.com> In-Reply-To: <504A16BA.7030407@mail.zedat.fu-berlin.de> References: <5049C13E.5010808@mail.zedat.fu-berlin.de> <504A0E46.3010306@FreeBSD.org> <504A16BA.7030407@mail.zedat.fu-berlin.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Re: FreeBSD 10.0-CURRENT: CLANG and port/clang weirdness! X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 16:46:07 -0000 > The LibreOffice package doesn't compile with the system's CLANG, so it > is installed whenever LibreOffice is installed. FWIW, it's using base clang and it does compile if ${OSVERSION} >= 900014 -- View this message in context: http://freebsd.1045724.n5.nabble.com/FreeBSD-10-0-CURRENT-CLANG-and-port-clang-weirdness-tp5741455p5741777.html Sent from the freebsd-current mailing list archive at Nabble.com. From owner-freebsd-current@FreeBSD.ORG Sat Sep 8 18:10:24 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 321E4106566B; Sat, 8 Sep 2012 18:10:24 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 8583F8FC08; Sat, 8 Sep 2012 18:10:22 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q88IAVkR075350; Sat, 8 Sep 2012 21:10:31 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q88IAJa4090888; Sat, 8 Sep 2012 21:10:19 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q88IAJv8090887; Sat, 8 Sep 2012 21:10:19 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 8 Sep 2012 21:10:19 +0300 From: Konstantin Belousov To: current@freebsd.org, amd64@freebsd.org Message-ID: <20120908181019.GK33100@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ws0tIAL3OFqxTKCX" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Subject: Small Ivy features: FSGSBASE and SMEP. X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 18:10:24 -0000 --ws0tIAL3OFqxTKCX Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Please find at http://people.freebsd.org/~kib/misc/smep.1.patch the patch which should enable the FSGSBASE and SMEP features supposedly present in the IvyBridge CPUs. FSGSBASE are four new instructions available in the 64bit mode only. They allow to access bases for %fs and %gs without touching MSRs. This makes it possible to both read and write bases in the user mode, or in ring 0 with lower overhead. At the moment, WRFSBASE/WRGSBASE instructions should work, but are useless since any interrupt or context switch overrides bases with the values set by the arch syscall. Still, RDFSBASE/RDGSBASE might be useful for some code and I see no reason not to enable them. SMEP is the nice feature of the processor which makes it trap if ring 0 tries to execute an instruction from usermode-accessible page. It is another mitigation for things like calling user-controllable function pointer in kernel, as well as a protection for NULL function pointer dereference. I am sure that we never execute anything in kernel from user page, but I did not tested the patch since I have no Ivy machine. I need your reports about boot on Ivy with patch applied. Please include the lines from verbose dmesg with CPU Features. In particular, the 'Standard Extended Features' report should appear in output. Thanks. --ws0tIAL3OFqxTKCX Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlBLigsACgkQC3+MBN1Mb4iU8wCeKHbqu15vuzYhJcrHq0O/TTF8 r6UAn3N+24R78Xenphvi4wF7kZzMVLqF =eUDd -----END PGP SIGNATURE----- --ws0tIAL3OFqxTKCX-- From owner-freebsd-current@FreeBSD.ORG Sat Sep 8 22:12:15 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D828B106564A for ; Sat, 8 Sep 2012 22:12:15 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id A6CCB8FC08 for ; Sat, 8 Sep 2012 22:12:15 +0000 (UTC) Received: by pbbrp2 with SMTP id rp2so1259656pbb.13 for ; Sat, 08 Sep 2012 15:12:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:from:date:message-id:subject:to:content-type; bh=Dmm82nzHqL4TfiRaOwPVpO5htj6CqFh4IKe383pDjPs=; b=gs/b8eZEJNRLDOD+QctXW4144TuHcX6AglwrxjrvA8AlcuerwHrHORkU0oAVCcVuZ/ zZF4XeLY0xnXplM9cNOJB0E27poT+9qtYn1LmoropgD2nnhu7b5/UN9Sejpp2epr0zop T+t59dJxQVZxkjjTxaqRxi7NBfvmZZVHCkQv8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type :x-gm-message-state; bh=Dmm82nzHqL4TfiRaOwPVpO5htj6CqFh4IKe383pDjPs=; b=ljnmMzh+9FSRWaIPJLmSyolrxxm8ESkITthLyYigavH0lnDfGWR8tyGak/VqwgEv80 Ax/dt5NclhD/VmGBFGg0zET9Bgooc/TRKRgYUZcOjkUeOWr1LtcTfIEG5opXOsalqoRb aBeqmbwEzgRU7aL3IqIVh6ug/pufJD1PAcErYiXEWV2LCl1AKGQA5dusuHKa3siU4Qs8 wJQsNEA5Ph5p25Q6yKehyMbSTa6mEwbyZOiASOgQ73wGn8k/0HYTnBNgVGNndnpYUMrb nnau5pGHJlJ0MNvVJ3s/PJBZl72VN0Zm4Ex21qQt9c73XT4FMs8ksLNH4ZdgU1fwtvb3 up/Q== Received: by 10.66.75.73 with SMTP id a9mr15508472paw.43.1347142335363; Sat, 08 Sep 2012 15:12:15 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.193.101 with HTTP; Sat, 8 Sep 2012 15:11:44 -0700 (PDT) From: Eitan Adler Date: Sat, 8 Sep 2012 18:11:44 -0400 Message-ID: To: current@freebsd.org, FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQkpHLs/fVgtvIarO2hCVbJBdv3Ra5ydH3YmobH3WcWSbRkv2oP/NjGtBqgppkhhlNuoHN9e Cc: Subject: heads up: conversation about removing CVS from HEAD on arch@ X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Sep 2012 22:12:16 -0000 Hi all, I started a thread about removing CVS from HEAD on arch. If you are interested in following or have something to say please use that list. -- Eitan Adler