From owner-freebsd-arch@FreeBSD.ORG Mon Oct 1 09:21:22 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35D8A106564A; Mon, 1 Oct 2012 09:21:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id A9A468FC0C; Mon, 1 Oct 2012 09:21:21 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q919LPnu014171; Mon, 1 Oct 2012 12:21:25 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q919LDt4016033; Mon, 1 Oct 2012 12:21:13 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q919LDdb016032; Mon, 1 Oct 2012 12:21:13 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 1 Oct 2012 12:21:13 +0300 From: Konstantin Belousov To: John Baldwin Message-ID: <20121001092113.GH35915@deviant.kiev.zoral.com.ua> References: <201209281847.39663.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Zlo2GrDIozf6aQFY" Content-Disposition: inline In-Reply-To: <201209281847.39663.jhb@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: arch@freebsd.org Subject: Re: stdio and short file descriptors revisited X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2012 09:21:22 -0000 --Zlo2GrDIozf6aQFY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Sep 28, 2012 at 06:47:39PM -0400, John Baldwin wrote: > Four years or so ago I cleaned up some of the stdio internals as fallout = =66rom=20 > running into problems with stdio using a short instead of an int to hold = file=20 > descriptors. Back then I got sidetracked with attempting to make FILE op= aque=20 > and ended up never getting around to bumping _file from a short to an int= =2E I=20 > recently ran back into the SHRT_MAX limit at work again and came up with = a=20 > patch to fix this. >=20 > To preserve the ABI, it is necessary to leave the existing short _file in= =20 > place and add a new int _file to the end of the FILE structure. Also, fo= r old=20 > applications, the old _file (_ofile in the patch) must still be valid. T= he=20 > approach I have taken is to bump the symbol version for routines that cre= ate=20 > FILE objects with a non-fake _file (fopen, fdopen, and freopen). The old= =20 > FBSD_1.0 variants still fail if an fd is greater than SHRT_MAX (and thus= =20 > cannot be safely stored in _ofile). The new FBSD_1.3 variants assign to = both=20 > _file and _ofile if the fd is less than SHRT_MAX. I also changed fileno() > to no longer be an inline macro in but to always be a function = call=20 > going forward. >=20 > If folks think this is ok, I'll hack up a modified version that hides _fi= le > from outside consumers (rename it to _nfile or some such) and send it for= a > ports-exp run before committing to make sure there aren't any 3rd party a= pps > accessing _file directly. >=20 > http://www.FreeBSD.org/~jhb/patches/stdio_file.patch The corner case left unhandled is the situation where we have a dso which is linked against FBSD_1.0 version of libc, but which gets FILE * as an API argument for some of its exported routines. If the implementation uses fileno(3), it would fail. I have no idea how to fix this, most likely, the issue is not fixable at all. Workaround seems to be to force the __isthreaded to 1. Might be, as an ugly hack, some flag could be added to the stdbuf(1), if anybody cares enough. Otherwise, the patch looks good. --Zlo2GrDIozf6aQFY Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlBpYIkACgkQC3+MBN1Mb4gCHwCg1XHzpsvs6LeerJaEtZXC87/x 0dEAoPOL19RkL5gOVOc0lvvzKTsMFjY+ =6tu+ -----END PGP SIGNATURE----- --Zlo2GrDIozf6aQFY-- From owner-freebsd-arch@FreeBSD.ORG Mon Oct 1 12:58:19 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 53179106564A for ; Mon, 1 Oct 2012 12:58:19 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 2893F8FC14 for ; Mon, 1 Oct 2012 12:58:19 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 3D2ADB958; Mon, 1 Oct 2012 08:58:18 -0400 (EDT) From: John Baldwin To: Konstantin Belousov Date: Mon, 1 Oct 2012 08:16:31 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p20; KDE/4.5.5; amd64; ; ) References: <201209281847.39663.jhb@freebsd.org> <20121001092113.GH35915@deviant.kiev.zoral.com.ua> In-Reply-To: <20121001092113.GH35915@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201210010816.31511.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 01 Oct 2012 08:58:18 -0400 (EDT) Cc: arch@freebsd.org Subject: Re: stdio and short file descriptors revisited X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2012 12:58:19 -0000 On Monday, October 01, 2012 5:21:13 am Konstantin Belousov wrote: > On Fri, Sep 28, 2012 at 06:47:39PM -0400, John Baldwin wrote: > > Four years or so ago I cleaned up some of the stdio internals as fallout from > > running into problems with stdio using a short instead of an int to hold file > > descriptors. Back then I got sidetracked with attempting to make FILE opaque > > and ended up never getting around to bumping _file from a short to an int. I > > recently ran back into the SHRT_MAX limit at work again and came up with a > > patch to fix this. > > > > To preserve the ABI, it is necessary to leave the existing short _file in > > place and add a new int _file to the end of the FILE structure. Also, for old > > applications, the old _file (_ofile in the patch) must still be valid. The > > approach I have taken is to bump the symbol version for routines that create > > FILE objects with a non-fake _file (fopen, fdopen, and freopen). The old > > FBSD_1.0 variants still fail if an fd is greater than SHRT_MAX (and thus > > cannot be safely stored in _ofile). The new FBSD_1.3 variants assign to both > > _file and _ofile if the fd is less than SHRT_MAX. I also changed fileno() > > to no longer be an inline macro in but to always be a function call > > going forward. > > > > If folks think this is ok, I'll hack up a modified version that hides _file > > from outside consumers (rename it to _nfile or some such) and send it for a > > ports-exp run before committing to make sure there aren't any 3rd party apps > > accessing _file directly. > > > > http://www.FreeBSD.org/~jhb/patches/stdio_file.patch > > The corner case left unhandled is the situation where we have a dso which > is linked against FBSD_1.0 version of libc, but which gets FILE * as an > API argument for some of its exported routines. If the implementation > uses fileno(3), it would fail. I have no idea how to fix this, most > likely, the issue is not fixable at all. Workaround seems to be to force > the __isthreaded to 1. Might be, as an ugly hack, some flag could be > added to the stdbuf(1), if anybody cares enough. > > Otherwise, the patch looks good. For things that layer on top of, say, funopen() (like libftpio, etc.), fileno() is meaningless. However, if libfoo.so calls fopen() and returns it to the caller, then yes, that would break in that if the opened fd was greater than 2^15 fileno() would return -1, not the real fd. At least it would return a known bogus fd such that attempts to use it should fail with EBADF rather than affecting some other unrelated fd. In practice I am not sure how many applications that use stdio really expect to be able to use fileno(). The two production cases I've seen that have run into the 2^15 limit haven't used fileno() at all, the breakage they see is that fopen() fails. Which is to say, I don't think that edge case will occur in practice. I do agree that it would be possible to add a workaround (such as forcing __isthreaded to true in the compat f*open() shims) in the future if it is really needed. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Oct 1 22:31:51 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 755E4106566C; Mon, 1 Oct 2012 22:31:51 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from exprod7og121.obsmtp.com (exprod7og121.obsmtp.com [64.18.2.20]) by mx1.freebsd.org (Postfix) with ESMTP id 416EE8FC14; Mon, 1 Oct 2012 22:31:49 +0000 (UTC) Received: from P-EMHUB03-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob121.postini.com ([64.18.6.12]) with SMTP ID DSNKUGoZ1ekWA20MmS+tkbQfzOTwPUxM0ev4@postini.com; Mon, 01 Oct 2012 15:31:51 PDT Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB03-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Mon, 1 Oct 2012 15:31:02 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id q91MV1h39000; Mon, 1 Oct 2012 15:31:01 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id E7D0D58093; Mon, 1 Oct 2012 15:31:00 -0700 (PDT) To: Garrett Cooper In-Reply-To: References: Comments: In-reply-to: Marcel Moolenaar message dated "Mon, 01 Oct 2012 12:17:52 -0700." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Mon, 1 Oct 2012 15:31:00 -0700 Message-ID: <20121001223100.E7D0D58093@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain Cc: freebsd-hackers@FreeBSD.org, freebsd-arch@FreeBSD.org, sjg@juniper.net Subject: Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2012 22:31:51 -0000 Hi Garrett, >> From: Garrett Cooper >> Subject: [CFT/RFC]: refactor bsd.prog.mk to understand multiple = >programs instead of a singular program >> Date: September 2, 2012 11:01:09 PM PDT >> To: freebsd-hackers@freebsd.org >> Cc: "freebsd-arch@FreeBSD.org Arch" >>=20 >> Hello, >> I've been a bit busy working on porting over ATF from NetBSD, and Thanks, we've been using ATF in Junos for a while and glad to see it being imported to FreeBSD. >> one of the pieces that's currently not available in FreeBSD that's >> available in NetBSD is the ability to understand and compile multiple >> programs. In order to do this I had to refactor bsd.prog.mk (a lot). A change like this to bsd.prog.mk can have considerable fallout. Eg. any makefile that tweaks OBJS is suddenly out of luck. Not to mention the fact that bsd.prog.mk goes from being relatively simple, to unspeakably hard to read, and all for rather limited return. Apart from ATF, is there any huge demand for building multiple progs in the same directory? FWIW we use progs.mk (as bsd.progs.mk) from ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz It isn't ideal, but it certainly avoids a lot of churn and complexity for what is essentially a corner case. We have an atf.test.mk which leverages that. It also handles automatically running the tests if building for the host. This allows us to fail the build if any unit-tests do not pass. Note, atf.test.mk is loosely derrived from NetBSD's bsd.tests.mk, but named for what it is (ATF specific tests) since we wanted the flexibility to have more than one test framework. --sjg From owner-freebsd-arch@FreeBSD.ORG Mon Oct 1 23:14:51 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A7EAB1065672; Mon, 1 Oct 2012 23:14:51 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6E9528FC0A; Mon, 1 Oct 2012 23:14:51 +0000 (UTC) Received: by padbi1 with SMTP id bi1so5329518pad.13 for ; Mon, 01 Oct 2012 16:14:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=CBDb5KN/i0qH0pY1Y4rPEUQWiLer88/3s3eZFuBshHY=; b=buB2kkAXTkStesPl5TJcWyHSnQQn8mEPDLuP6iB61E5WkQyrzmdN0M3QrTrELNiURx y3lysYxRNbRr2YDrz7XEB6bXLGUbGnhhBOqWzpulaXXE/k7s+kSZQ8z7xaNGK3qLBJe2 onUHrrgO3yX/lvItz1G3X/jtrhwxyvgjbQVUndZDMRWNU/7iFZ4QF8u1vgnlO5ZnMTO7 wXWY8AqA9BNWg99NFh3HihlREMVdUH7AW6zx1l/UUJvtnnKnFsIqWiXWY2/k1ooeqdA6 jxF+sfMcW4kE4bZJ8nV/LwFTWGp3rz6jacKCkWqqaVw4WS/iMVwGE/N2HGz4Fwq1bUdd ge4g== Received: by 10.66.88.233 with SMTP id bj9mr39656786pab.72.1349133290901; Mon, 01 Oct 2012 16:14:50 -0700 (PDT) Received: from fuji-wireless.local (c-24-19-191-56.hsd1.wa.comcast.net. [24.19.191.56]) by mx.google.com with ESMTPS id py9sm10951798pbb.20.2012.10.01.16.14.48 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 01 Oct 2012 16:14:49 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=windows-1252 From: Garrett Cooper In-Reply-To: <20121001223100.E7D0D58093@chaos.jnpr.net> Date: Mon, 1 Oct 2012 16:15:16 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20121001223100.E7D0D58093@chaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1278) Cc: freebsd-hackers@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2012 23:14:51 -0000 Hi Simon! On Oct 1, 2012, at 3:31 PM, Simon J. Gerraty wrote: > Hi Garrett, >=20 >>> From: Garrett Cooper >>> Subject: [CFT/RFC]: refactor bsd.prog.mk to understand multiple =3D >> programs instead of a singular program >>> Date: September 2, 2012 11:01:09 PM PDT >>> To: freebsd-hackers@freebsd.org >>> Cc: "freebsd-arch@FreeBSD.org Arch" >>> =3D20 >>> Hello, >>> I've been a bit busy working on porting over ATF from NetBSD, and >=20 > Thanks, we've been using ATF in Junos for a while and glad to see it=20= > being imported to FreeBSD. Thanks (and I appreciate the help marcel@ has given by taking the first = step in importing it). >>> one of the pieces that's currently not available in FreeBSD that's >>> available in NetBSD is the ability to understand and compile = multiple >>> programs. In order to do this I had to refactor bsd.prog.mk (a lot). >=20 > A change like this to bsd.prog.mk can have considerable fallout. > Eg. any makefile that tweaks OBJS is suddenly out of luck. Well=85 any Makefiles attempting to be "clever" (e.g. crunchgen) are out = of luck and would need to change. Most [include] Makefiles will see = virtually no change. > Not to mention the fact that bsd.prog.mk goes from being relatively > simple, to unspeakably hard to read, and all for rather limited = return.=20 I sort of tried to match bsd.prog.mk in NetBSD in the beginning, but = things diverged from there=85 I'm sure things could be made more = readable so any comments would be much appreciated :)! > Apart from ATF, is there any huge demand for building multiple progs = in > the same directory? Well, I noticed that there are a couple ugly messes in the gnu/... = directory that attempt to work around the fact that bsd.prog.mk doesn't = support more than one program by being tricky and emulating bsd.prog.mk = instead of solving the generic problem (and once I got over that = compatibility issue I stopped tracking this class of consumer). Most = consumers don't care (they split up programs into different directories = or use hardlink hacks/basename detection to differentiate one program = functionally from another). Getting back to the idea at hand, the enhancement goal was to get the = testcases to live [and optionally be built/installed] with the source = code to avoid bitrot; I know this isn't the current NetBSD design, but = this is what was requested be done by gnn, marcel, and mdf, and I agree. = It order to do this, I need to be able to build multiple programs so = things are as transparent. On the flipside, bsd.prog[s].mk can live on = its own, be pulled in automatically by bsd.test.mk, and that would be = it. This encourages code duplication though and bugs can persist in = either Makefile, when I'd rather work out all the kinks in whatever = succeeds the legacy bsd.prog.mk file. > FWIW we use progs.mk (as bsd.progs.mk) from > ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz=20 > It isn't ideal, but it certainly avoids a lot of churn and complexity > for what is essentially a corner case. This requires pulling bmake into FreeBSD proper in order for things to = function the last I checked as bsd.prog.mk depends upon bmake directives = in order to function properly 100% of the time (and there are external = dependencies and assumptions one has to deal with when using = bsd.prog.mk, etc from NetBSD that made that a bit of a no-go without = refactoring/pulling in bits from NetBSD). I could be wrong though, so = please let me know if I am :). Ideally however, I would like doing this compared to running a custom = build infrastructure, but (as you probably know) this requires = rototilling the current FreeBSD build system to a large degree = (definitely not impossible=85 just needs time and effort). > We have an atf.test.mk which leverages that. > It also handles automatically running the tests if building for the > host. This allows us to fail the build if any unit-tests do not pass. Hmm=85 that wasn't really the end-goal of bsd.test.mk based on my = reading, but I'm sure Julio would be interested in it. I need to do = adjusting in order to allow automatically executing testcases compatible = to the host architecture, but that isn't hard to do (no more than a = couple hours work). > Note, atf.test.mk is loosely derrived from NetBSD's bsd.tests.mk, but > named for what it is (ATF specific tests) since we wanted the > flexibility to have more than one test framework. bsd.test.mk in NetBSD isn't completely test-agnostic, but it isn't = entirely ATF centric either. I'm trying not to stray too far from NetBSD = in this area because there really isn't any reason to do so. FWIW I'd = like to see other test frameworks (lua, unittest//nose, etc) just snap = into bsd.test.mk as easily as possible as it would make some groups = lives easier, but that requires a bit more thought for another day. Thanks for the feedback! -Garrett= From owner-freebsd-arch@FreeBSD.ORG Tue Oct 2 00:02:12 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3FE6C1065674; Tue, 2 Oct 2012 00:02:12 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from exprod7og110.obsmtp.com (exprod7og110.obsmtp.com [64.18.2.173]) by mx1.freebsd.org (Postfix) with ESMTP id 978518FC0C; Tue, 2 Oct 2012 00:02:10 +0000 (UTC) Received: from P-EMHUB03-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob110.postini.com ([64.18.6.12]) with SMTP ID DSNKUGovAWxyWzC4IQvD1sZjrFmUuVrQOwO3@postini.com; Mon, 01 Oct 2012 17:02:11 PDT Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB03-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Mon, 1 Oct 2012 17:00:32 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id q9200Uh40887; Mon, 1 Oct 2012 17:00:31 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 54CEE58093; Mon, 1 Oct 2012 17:00:30 -0700 (PDT) To: Garrett Cooper In-Reply-To: References: <20121001223100.E7D0D58093@chaos.jnpr.net> Comments: In-reply-to: Garrett Cooper message dated "Mon, 01 Oct 2012 16:15:16 -0700." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Mon, 1 Oct 2012 17:00:30 -0700 Message-ID: <20121002000030.54CEE58093@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain Cc: freebsd-hackers@FreeBSD.org, freebsd-arch@FreeBSD.org, sjg@juniper.net Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 00:02:12 -0000 >> Not to mention the fact that bsd.prog.mk goes from being relatively >> simple, to unspeakably hard to read, and all for rather limited = >return. This btw I think is the more important issue. I was looking at bsd.prog.mk in netbsd the other day. It has no business being that complex. >Getting back to the idea at hand, the enhancement goal was to get the = >testcases to live [and optionally be built/installed] with the source = >code to avoid bitrot; I know this isn't the current NetBSD design, but = >this is what was requested be done by gnn, marcel, and mdf, and I agree. = I agree, that's what we do too. >It order to do this, I need to be able to build multiple programs so = >things are as transparent. On the flipside, bsd.prog[s].mk can live on = We put the test cases in a subdir of the lib/prog This has multiple benefits, and eliminates any impact on the normal build of said libs/progs. Also a number of our teams find it necessary to create mock classes etc to adequately test their libs. Keeping all this in a tests/ subdir makes it easy, to keep things neat & up to date, and ensure that the tests pass. Trying to do all that in the same dir as the lib would be ugly. >> FWIW we use progs.mk (as bsd.progs.mk) from >> ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz=20 >> It isn't ideal, but it certainly avoids a lot of churn and complexity >> for what is essentially a corner case. > >This requires pulling bmake into FreeBSD proper in order for things to = >function the last I checked as bsd.prog.mk depends upon bmake directives = This is already happening ;-) >Ideally however, I would like doing this compared to running a custom = >build infrastructure, but (as you probably know) this requires = >rototilling the current FreeBSD build system to a large degree = Actually building FreeBSD-current using bmake requires only modest changes. >> We have an atf.test.mk which leverages that. >> It also handles automatically running the tests if building for the >> host. This allows us to fail the build if any unit-tests do not pass. > >Hmm=85 that wasn't really the end-goal of bsd.test.mk based on my = >reading, I know, but it is a very useful thing to be able to do. You can add knobs to controll such things of course. >bsd.test.mk in NetBSD isn't completely test-agnostic, but it isn't = >entirely ATF centric either. I'm trying not to stray too far from NetBSD = >in this area because there really isn't any reason to do so. FWIW I'd = Sure - we imported ATF directly from NetBSD to minimize the churn and not had to change much at all. bsd.test.mk was a point worth diverging on though. >like to see other test frameworks (lua, unittest//nose, etc) just snap = >into bsd.test.mk as easily as possible as it would make some groups = Yes, but making one test.mk handle potentially lots of frameworks will get rather ugly quite quickly. Better to add per-framework .mk files, and perhaps have them include a bsd.test.mk which only handles high level logic rather than the details of the frameworks. That's laregly why we didn't use bsd.test.mk --sjg From owner-freebsd-arch@FreeBSD.ORG Tue Oct 2 12:40:53 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E4F91065670; Tue, 2 Oct 2012 12:40:53 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 00C1E8FC0C; Tue, 2 Oct 2012 12:40:53 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 67421B911; Tue, 2 Oct 2012 08:40:52 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Tue, 2 Oct 2012 07:50:23 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p20; KDE/4.5.5; amd64; ; ) References: <20121001223100.E7D0D58093@chaos.jnpr.net> In-Reply-To: <20121001223100.E7D0D58093@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201210020750.23358.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 02 Oct 2012 08:40:52 -0400 (EDT) Cc: Garrett Cooper , freebsd-hackers@freebsd.org, "Simon J. Gerraty" Subject: Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 12:40:53 -0000 On Monday, October 01, 2012 6:31:00 pm Simon J. Gerraty wrote: > Hi Garrett, > > >> From: Garrett Cooper > >> Subject: [CFT/RFC]: refactor bsd.prog.mk to understand multiple = > >programs instead of a singular program > >> Date: September 2, 2012 11:01:09 PM PDT > >> To: freebsd-hackers@freebsd.org > >> Cc: "freebsd-arch@FreeBSD.org Arch" > >>=20 > >> Hello, > >> I've been a bit busy working on porting over ATF from NetBSD, and > > Thanks, we've been using ATF in Junos for a while and glad to see it > being imported to FreeBSD. > > >> one of the pieces that's currently not available in FreeBSD that's > >> available in NetBSD is the ability to understand and compile multiple > >> programs. In order to do this I had to refactor bsd.prog.mk (a lot). > > A change like this to bsd.prog.mk can have considerable fallout. > Eg. any makefile that tweaks OBJS is suddenly out of luck. > > Not to mention the fact that bsd.prog.mk goes from being relatively > simple, to unspeakably hard to read, and all for rather limited return. > > Apart from ATF, is there any huge demand for building multiple progs in > the same directory? > > FWIW we use progs.mk (as bsd.progs.mk) from > ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz > It isn't ideal, but it certainly avoids a lot of churn and complexity > for what is essentially a corner case. This sounds like a superior approach. It doesn't break any current use cases while giving the ability to build multiple programs in the few places that need it. It sounds like there are a few places under gnu/ from Garrett's reply that might be able to make use of this as well. BTW, one general comment. There seem to be two completely independent groups of folks working on ATF (e.g. there have been two different imports of ATF into the tree in two different locations IIRC, and now we have two different sets of patches to our system makefiles). Are these two groups talking to each other at all? I know in May that many folks (certainly multiple vendors) are interested in ATF, and it seems that both Juniper and Isilon have ported ATF internally. It seems that it might be good for the two groups to work together to avoid stomping on each other's toes. It seems there are some differences in the two approaches that merit working out to avoid a lot of wasted effort on both sides. Do we already have a freebsd-atf@ mailing list? If not, perhaps we should create one and start these discussions there? -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Oct 2 14:19:56 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C5139106566C; Tue, 2 Oct 2012 14:19:56 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6E7048FC12; Tue, 2 Oct 2012 14:19:56 +0000 (UTC) Received: by oagn9 with SMTP id n9so6298531oag.13 for ; Tue, 02 Oct 2012 07:19:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=8qvwGc8uFPh6HD2amh6kNVgUUS/wI4OOAWBrKkyGk98=; b=CmMMZxxkO/bTiMulW8aiK8tWOwyKE//g0TKx9TaWN+CFs62js3fF5kCcsBUctod2KU F7Kb22ti53Fx5jx3FsYhjchShaO8JRo4I5m3PNJEaAjtcQsgUWS+z6nEtzF7+mE5j6+p yPGJpfOLuGBjcZtstm532ODKnDDe1GOPtkn/4NRS2xm8x2gSqeS3RrxWYAUGjuCL+vjH n40YjTVdXDaDsiSPWGZR/OpcVyfbTZYb0L5uJntXMi4fOeUEZjknWdd49oNID4dodW1V dP64oScFp6KWIBh49tk9vtZpgcWPAMubhOYU3J/o62tdDegWZtVzq+xhUv6nCkdk+OSd wzmw== MIME-Version: 1.0 Received: by 10.60.170.241 with SMTP id ap17mr14742956oec.4.1349187595757; Tue, 02 Oct 2012 07:19:55 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Tue, 2 Oct 2012 07:19:55 -0700 (PDT) In-Reply-To: <20121002000030.54CEE58093@chaos.jnpr.net> References: <20121001223100.E7D0D58093@chaos.jnpr.net> <20121002000030.54CEE58093@chaos.jnpr.net> Date: Tue, 2 Oct 2012 07:19:55 -0700 Message-ID: From: Garrett Cooper To: "Simon J. Gerraty" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 14:19:56 -0000 On Mon, Oct 1, 2012 at 5:00 PM, Simon J. Gerraty wrote: >>> Not to mention the fact that bsd.prog.mk goes from being relatively >>> simple, to unspeakably hard to read, and all for rather limited = >>return. > > This btw I think is the more important issue. > I was looking at bsd.prog.mk in netbsd the other day. > It has no business being that complex. Ok. Even though I originally thought that my changes were a bit hacky for not modifying bsd.man.mk, bsd.nls.mk, etc, looking back the way they were resolved was a little elegant in its simplicity (albeit not optimized). ... >>It order to do this, I need to be able to build multiple programs so = >>things are as transparent. On the flipside, bsd.prog[s].mk can live on = > > We put the test cases in a subdir of the lib/prog > This has multiple benefits, and eliminates any impact on the normal > build of said libs/progs. Hmmm... that's one of the 3 approaches I provided, but it turned out to be annoying to make this work at first inspection because of how things were done in our build system. Just for review (even though it's OT), the three approaches I presented were as follows... Note: in all 3 examples, clock.c is the source code and t_clock.c is the relevant unittest code. 1. Test programs live with the sources (this was the requested approach), e.g. lib/libc/gen/... .../clock.c .../t_clock.c 2. Test programs live in subdirs: lib/libc/gen/... .../clock.c .../tests/... .../t_clock.c 3. Test programs completely decoupled from the source tree: lib/libc/gen/... .../clock.c tests/lib/libc/gen/... .../t_clock.c A hybrid approach (1.+3. / 2.+3.) should probably be used anyhow for stress tests and the like that really have no business living in one particular directory in the source tree (sort of achieving what tools/regression does today, but hopefully in a less messy manner as tools/regression appears to have grown organically minus any single sane order). > Also a number of our teams find it necessary to create mock classes etc to > adequately test their libs. Keeping all this in a tests/ subdir makes > it easy, to keep things neat & up to date, and ensure that the tests > pass. Trying to do all that in the same dir as the lib would be ugly. This (consolidating everything under a single directory) is the way that was requested by the beforementioned parties. >>> FWIW we use progs.mk (as bsd.progs.mk) from >>> ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz=20 >>> It isn't ideal, but it certainly avoids a lot of churn and complexity >>> for what is essentially a corner case. >> >>This requires pulling bmake into FreeBSD proper in order for things to = >>function the last I checked as bsd.prog.mk depends upon bmake directives = > > This is already happening ;-) I wish I had known that. I pinged marcel/obrien about this some weeks ago and didn't receive a response. Originally bsd.progs.mk was going to be a home grown effort that was going to be killed off and replaced with NetBSD's build infrastructure, but I didn't get a reply and had to put this (my work to replace bsd.prog.mk) plan in motion given requests I was provided above for resolving the source/unittest code consolidation effort. >>Ideally however, I would like doing this compared to running a custom = >>build infrastructure, but (as you probably know) this requires = >>rototilling the current FreeBSD build system to a large degree = > > Actually building FreeBSD-current using bmake requires only modest changes. Yes for the most part, and I agree that bmake is definitely more advanced than FreeBSD pmake so I consider it a very welcome change. What are the plans for importing bmake into FreeBSD (this should probably be a different thread)? ... > I know, but it is a very useful thing to be able to do. > You can add knobs to control such things of course. I agree that it's a good thing (in theory) -- it'd just help to discuss this with Julio first because I know of a couple cases where this would be unusable given how bsd.test.mk is coded: 1. The "it works for me on my machine!" certification: It encourages environment tainting, which could be a really, really bad thing; I've seen developers take interesting shortcuts when testing their code (me included sometimes :)..); I've seen hardcoded paths, harcoded "paths" for named semaphores, things that "just work" because of someone's host environment, feature specific assumptions (developer X was doing testing on feature Y and things broke when he/she committed the testcase to head), etc where the "it works for me on my machine!" certification would be true more often than not. These same individuals would more likely than not execute things taking shortcuts, but I want to avoid creating a system where developers cut corners and commit too early because working within the confines of the system is not conducive to getting work done. 2. Failure in a subdirectory results in lost coverage: a/... .../b/... <- tests live here .../c/d/... <- tests live here If I execute make test from a/ (one of my goals), then it should iterate down a/b, then a/c/d and run the tests in a DFS style (BFS if invoked with -j > 1 -- AIIIEEEE). If a/b fails to execute the make process will bomb out because of how it executes shell statements. However, if I let a/b pass, then developers/testers will accidentally overlook failures, unless output is captured from the run and errors are searched for in the end. This isn't a super big problem if the tests are deterministic enough that fixing one test permits the others to run, but the lost test coverage in that period of time (or others if it transiently fails) could disguise other bugs. This would be fixed by changing bsd.test.mk to use atf-run properly, but that's work (not hard) that still needs to be done. (there are some other cases, but they're eluding me now) > Yes, but making one test.mk handle potentially lots of frameworks > will get rather ugly quite quickly. Understood and I agree. I wasn't advocating having a single, monolithic bsd.test.mk with all the test frameworks integrated into it (that would be silly and unmaintainable). > Better to add per-framework .mk files, and perhaps have them include a > bsd.test.mk which only handles high level logic rather than the details > of the frameworks. > > That's laregly why we didn't use bsd.test.mk Hmmmm... My goal was to make ATF a "one ring to rule them all" infrastructure so that way everything could be unified under ATF in one way, shape or form (even if it's just reporting), because lord knows we don't need a lua, unittest/nose, etc wrapper for reporting results. All of the jobs I've worked at write custom wrappers around testing infrastructures in order to present the data in a pretty format (HTML), but that's a largely wasted effort (unless someone REALLY wants to learn CSS/HTML/JS, which is an ongoing pain thanks to browser incompatibilities), but I realize that this is also more effort than I could get to right away, and it would have to be done as needed instead. Thanks! -Garrett From owner-freebsd-arch@FreeBSD.ORG Tue Oct 2 14:29:50 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A32AE106566B; Tue, 2 Oct 2012 14:29:50 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3D2808FC08; Tue, 2 Oct 2012 14:29:49 +0000 (UTC) Received: by obbwc20 with SMTP id wc20so6762440obb.13 for ; Tue, 02 Oct 2012 07:29:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=yMr25PEYOSjTSQbPntob1x4MGkqjurBBoNIbBfEp8ts=; b=W9/WfWyAB9+Z0jZc73axkCamQGvmnokjjnKbbt9o0aESQR1N/GXYSAKAsO2+piib6X 8aukgtyYcbo3tgGX4+BlEiVaN808Gdj64LuSX9zzXVK8tIhYjDxfeqwiRyv9eGnWVTPT sicfD+HEFaAz2ZthZ88tZ/zbnMmVwQG2KIPZIE58Ej8yAQlXr7ZAr+g5i2d0ho0uXWOa 3lw6Ltm98Fj5WM2HN8PJjge2hwXMojOTe4bXZ8D4JSP6njw1h2SYi8EEs7MnTaJQsZCz AUbKJStFLAOKEBkkjkqjJuN8XGzLDjU/oQe9/UO4Xb2VpGX7E+8zTSxAzwH9DBzvq3Hz wilw== MIME-Version: 1.0 Received: by 10.182.231.66 with SMTP id te2mr5485610obc.67.1349188189334; Tue, 02 Oct 2012 07:29:49 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Tue, 2 Oct 2012 07:29:49 -0700 (PDT) In-Reply-To: <201210020750.23358.jhb@freebsd.org> References: <20121001223100.E7D0D58093@chaos.jnpr.net> <201210020750.23358.jhb@freebsd.org> Date: Tue, 2 Oct 2012 07:29:49 -0700 Message-ID: From: Garrett Cooper To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, "Simon J. Gerraty" , freebsd-arch@freebsd.org Subject: Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 14:29:50 -0000 On Tue, Oct 2, 2012 at 4:50 AM, John Baldwin wrote: ... > This sounds like a superior approach. It doesn't break any current use > cases while giving the ability to build multiple programs in the few > places that need it. It sounds like there are a few places under gnu/ > from Garrett's reply that might be able to make use of this as well. For the record, gnu/cc/cc_tools/Makefile is where I first spotted a potential "bsd.progs.mk" candidate. Most of the other code doesn't care given how things are organized in our source tree. > BTW, one general comment. There seem to be two completely independent > groups of folks working on ATF (e.g. there have been two different > imports of ATF into the tree in two different locations IIRC, and now > we have two different sets of patches to our system makefiles). > > Are these two groups talking to each other at all? I know in May that > many folks (certainly multiple vendors) are interested in ATF, and it > seems that both Juniper and Isilon have ported ATF internally. It seems > that it might be good for the two groups to work together to avoid > stomping on each other's toes. It seems there are some differences in > the two approaches that merit working out to avoid a lot of wasted > effort on both sides. Both parties (Isilon/Juniper) are converging on the ATF porting work that Giorgos/myself have done after talking at the FreeBSD Foundation meet-n-greet. I have contributed all of the patches that I have other to marcel for feedback. > Do we already have a freebsd-atf@ mailing list? If not, perhaps we > should create one and start these discussions there? Probably wouldn't be a bad idea as I'm currently suspended a bit waiting on feedback for how to proceed; too bad freebsd-test is being used for other things :).. Thanks! -Garrett From owner-freebsd-arch@FreeBSD.ORG Tue Oct 2 15:39:21 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F192D106564A; Tue, 2 Oct 2012 15:39:20 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from exprod7og118.obsmtp.com (exprod7og118.obsmtp.com [64.18.2.8]) by mx1.freebsd.org (Postfix) with ESMTP id A385B8FC15; Tue, 2 Oct 2012 15:39:19 +0000 (UTC) Received: from P-EMHUB03-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob118.postini.com ([64.18.6.12]) with SMTP ID DSNKUGsKob/1aHEXgvXZAzyGrIgFecKw5o35@postini.com; Tue, 02 Oct 2012 08:39:20 PDT Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB03-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Tue, 2 Oct 2012 08:38:13 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id q92FcDh48007; Tue, 2 Oct 2012 08:38:13 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id D766B58093; Tue, 2 Oct 2012 08:38:12 -0700 (PDT) To: John Baldwin In-Reply-To: <201210020750.23358.jhb@freebsd.org> References: <20121001223100.E7D0D58093@chaos.jnpr.net> <201210020750.23358.jhb@freebsd.org> Comments: In-reply-to: John Baldwin message dated "Tue, 02 Oct 2012 07:50:23 -0400." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Tue, 2 Oct 2012 08:38:12 -0700 Message-ID: <20121002153812.D766B58093@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain Cc: Garrett Cooper , freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org, sjg@juniper.net Subject: Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 15:39:21 -0000 On Tue, 2 Oct 2012 07:50:23 -0400, John Baldwin writes: >BTW, one general comment. There seem to be two completely independent >groups of folks working on ATF (e.g. there have been two different >imports of ATF into the tree in two different locations IIRC, and now >we have two different sets of patches to our system makefiles). Yes, and no. We (Juniper) have been using ATF for some time, and were going to do import etc, but for various reasons haven't done it yet. In part I guess because having bmake in tree would have made things much simpler - avoiding re-inventing wheels. Garrett, has forged ahead, and we are fine with that - Marcel did the import for him. Obviously though (as I've probably just made clear) we don't want ATF to unnecessarily complicate the build. We hope to get the initial bmake commit done this week, and then we can help Garrett get ATF going with minimal fuss. >Are these two groups talking to each other at all? Yes, but I don't think Garrett was aware of the bmake work. > It seems there are some differences in >the two approaches that merit working out to avoid a lot of wasted >effort on both sides. The differences are probably very minor, and hopefully easily sorted out. The most significant being how to build the multiple test apps in one directory. Related to that is the exact location. I believe we are all agreed that tests should be co-located with the code they exercise - to facilitate testing as you make changes. We use a tests/ subdir per lib and prog that has unit-tests and I would recommend doing the same. This is consistent with the above goal, and the separate directory is very useful for keeping the build simple - eg. libfoo/Makefile can continue to just use bsd.lib.mk while libfoo/tests/Makefile can use any of bsd.prog.mk, bsd.progs.mk, bsd.test.mk or atf.test.mk Also, these separate dirs become even more important when using meta mode because you want the all/default target to "just do the right thing", and do it the same way every time, to avoid churn in dependencies. >Do we already have a freebsd-atf@ mailing list? If not, perhaps we >should create one and start these discussions there? Don't know, but fine either way. --sjg From owner-freebsd-arch@FreeBSD.ORG Tue Oct 2 16:19:03 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7714D106566B; Tue, 2 Oct 2012 16:19:03 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from exprod7og124.obsmtp.com (exprod7og124.obsmtp.com [64.18.2.26]) by mx1.freebsd.org (Postfix) with ESMTP id 0CC958FC15; Tue, 2 Oct 2012 16:19:01 +0000 (UTC) Received: from P-EMHUB03-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob124.postini.com ([64.18.6.12]) with SMTP ID DSNKUGsT71yeK0mQ0bP/uNjtntajTyvZOkML@postini.com; Tue, 02 Oct 2012 09:19:03 PDT Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB03-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Tue, 2 Oct 2012 09:17:40 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id q92GHeh43108; Tue, 2 Oct 2012 09:17:40 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id EFE0E58093; Tue, 2 Oct 2012 09:17:39 -0700 (PDT) To: Garrett Cooper In-Reply-To: References: <20121001223100.E7D0D58093@chaos.jnpr.net> <20121002000030.54CEE58093@chaos.jnpr.net> Comments: In-reply-to: Garrett Cooper message dated "Tue, 02 Oct 2012 07:19:55 -0700." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Tue, 2 Oct 2012 09:17:39 -0700 Message-ID: <20121002161739.EFE0E58093@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain Cc: freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org, sjg@juniper.net Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 16:19:03 -0000 On Tue, 2 Oct 2012 07:19:55 -0700, Garrett Cooper writes: >> We put the test cases in a subdir of the lib/prog >> This has multiple benefits, and eliminates any impact on the normal >> build of said libs/progs. > >Hmmm... that's one of the 3 approaches I provided, but it turned out >to be annoying to make this work at first inspection because of how >things were done in our build system. Just for review (even though >it's OT), the three approaches I presented were as follows... > >Note: in all 3 examples, clock.c is the source code and t_clock.c is >the relevant unittest code. There is actually another: lib/libc/Makefile lib/libc/gen/... .../clock.c .../t_clock.c lib/libc/tests/Makefile that is the makefile for building/running the tests lives in the subdir, there are advantages to this, but the test code itself can be anywhere you like. Either next to the code that it tests, or in the tests/ subdir, another subdir, or whatever combination makes the most sense. Most of our ATF users put their test code in the subdir fwiw. >> Also a number of our teams find it necessary to create mock classes etc to >> adequately test their libs. Keeping all this in a tests/ subdir makes >> it easy, to keep things neat & up to date, and ensure that the tests >> pass. Trying to do all that in the same dir as the lib would be ugly. > >This (consolidating everything under a single directory) is the way >that was requested by the beforementioned parties. Sorry for not participating in that discussion ;-) John - perhaps we do need that ATF list? For example building a library and test apps in the one directory either requires turning bsd.lib.mk into an even more complex thing that a multi-prog bsd.pog.mk, or having the makefile behave completely differently depending on the target requested. That later is a bad idea. It precludes being able to seamlessly integrate unit-tests into the build, and would need to be fixed if/when attempting to introduce meta mode. Typically the unit-tests depend on the library they are testing, a separate subdir for the tests/Makefile makes capturing that dependency easy - otherwise you are back to the unspeakably complex bsd.lib.mk Far better to get these things right first up. >I wish I had known that. I pinged marcel/obrien about this some weeks >ago and didn't receive a response. Many appologies. >What are the plans for importing bmake into FreeBSD (this should >probably be a different thread)? The import has been done - just needs to be merged to contrib, and there's a small patch to be committed. We hope to get that done this week. >I agree that it's a good thing (in theory) -- it'd just help to >discuss this with Julio first because I know of a couple cases where >this would be unusable given how bsd.test.mk is coded: I do talk to Julio about ATF (I'm sjg@netbsd.org ;-) though not specifically this - beyond (I think) mentioning that I did it differently. >1. The "it works for me on my machine!" certification: > >It encourages environment tainting, which could be a really, really >bad thing; I've seen developers take interesting shortcuts when True. In our (Junos) build we cleanse the environment rather carefully and have a very clear distinction between building for the "host" (which may just happend to be i386 based) and the i386 "target" for example. I can provide build smarts to do all that - but didn't want to shove it down anyone's throat ;-) What "works for us" doesn't necessarily work for everyone. >2. Failure in a subdirectory results in lost coverage: > >a/... > .../b/... <- tests live here > .../c/d/... <- tests live here > >If I execute make test from a/ (one of my goals), then it should >iterate down a/b, then a/c/d and run the tests in a DFS style (BFS if This gets back to the bmake topic. We actually avoid "walking" the tree at all, eg. in the Junos build I have a target "atf-tests" which builds all ATF tests dirs regardless of where they are (and any pre-requisits they have). This is handy to include as a dependency of the build target we use for branch syncs etc, to ensure no unit-tests atrophy. But the most common case - and the one to optimize for, is the person making changes to libfoo, testing that he hasn't broken anything. >could disguise other bugs. This would be fixed by changing bsd.test.mk >to use atf-run properly, but that's work (not hard) that still needs >to be done. Yes, atf.test.mk already does that. >Hmmmm... My goal was to make ATF a "one ring to rule them all" >infrastructure so that way everything could be unified under ATF in ATF is a pretty good framework - which is why we (Juniper) use it. We want the flexibility to have more than one framework, but the number should be very small. So far I have persuaded teams that wanted to adopt alternate frameworks, that ATF could easily meet their needs. >one way, shape or form (even if it's just reporting), because lord >knows we don't need a lua, unittest/nose, etc wrapper for reporting >results. All of the jobs I've worked at write custom wrappers around Yes, having all "frameworks" able to output results in a consistent format is definitely a good thing. Thanks --sjg From owner-freebsd-arch@FreeBSD.ORG Tue Oct 2 18:16:49 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 94AFD106564A; Tue, 2 Oct 2012 18:16:49 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 66C558FC12; Tue, 2 Oct 2012 18:16:49 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id C10C6B911; Tue, 2 Oct 2012 14:16:48 -0400 (EDT) From: John Baldwin To: Garrett Cooper Date: Tue, 2 Oct 2012 10:37:27 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p20; KDE/4.5.5; amd64; ; ) References: <201210020750.23358.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201210021037.27762.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 02 Oct 2012 14:16:48 -0400 (EDT) Cc: freebsd-hackers@freebsd.org, "Simon J. Gerraty" , freebsd-arch@freebsd.org Subject: Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Oct 2012 18:16:49 -0000 On Tuesday, October 02, 2012 10:29:49 am Garrett Cooper wrote: > On Tue, Oct 2, 2012 at 4:50 AM, John Baldwin wrote: > > ... > > > This sounds like a superior approach. It doesn't break any current use > > cases while giving the ability to build multiple programs in the few > > places that need it. It sounds like there are a few places under gnu/ > > from Garrett's reply that might be able to make use of this as well. > > For the record, gnu/cc/cc_tools/Makefile is where I first spotted a > potential "bsd.progs.mk" candidate. Most of the other code doesn't > care given how things are organized in our source tree. > > > BTW, one general comment. There seem to be two completely independent > > groups of folks working on ATF (e.g. there have been two different > > imports of ATF into the tree in two different locations IIRC, and now > > we have two different sets of patches to our system makefiles). > > > > Are these two groups talking to each other at all? I know in May that > > many folks (certainly multiple vendors) are interested in ATF, and it > > seems that both Juniper and Isilon have ported ATF internally. It seems > > that it might be good for the two groups to work together to avoid > > stomping on each other's toes. It seems there are some differences in > > the two approaches that merit working out to avoid a lot of wasted > > effort on both sides. > > Both parties (Isilon/Juniper) are converging on the ATF porting work > that Giorgos/myself have done after talking at the FreeBSD Foundation > meet-n-greet. I have contributed all of the patches that I have other > to marcel for feedback. This is very non-obvious to the public at large (e.g. there was no public response to one group's inquiry about the second ATF import for example). Also, given that you had no idea that sgf@ and obrien@ were working on importing NetBSD's bmake as a prerequisite for ATF, it seems that whatever discussions were held were not very detailed at best. I think it would be good to have the various folks working on ATF to at least summarize the current state of things and sketch out some sort of plan or roadmap for future work in a public forum (such as atf@, though a summary mail would be quite appropriate for arch@). -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Oct 3 13:06:45 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E2AEC106564A for ; Wed, 3 Oct 2012 13:06:45 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 415198FC12 for ; Wed, 3 Oct 2012 13:06:44 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA06150 for ; Wed, 03 Oct 2012 16:06:37 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <506C385C.3020400@FreeBSD.org> Date: Wed, 03 Oct 2012 16:06:36 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120911 Thunderbird/15.0.1 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 13:06:46 -0000 Currently we produce "slightly" different binaries for x86 boot code depending whether MACHINE_CPUARCH is i386 or amd64. I think that there is no good reason for this, since in both cases we use exactly the same code and target the same classes of machines. In other words, the binaries should be interchangeable[*]. The difference boils down to using -march=i386 on amd64 while i386 uses default compiler flags, which are equivalent to -march=i486 -mtune=generic. If my analysis is correct, the only thing affected by the flags in the boot code is use of leave instruction when -Os is _not_ specified. For -march=i386 our gcc prefers using leave. For -march=i486 it thinks that movs+pops are faster than leave and so prefers to not use it. If -Os is specified, then leave is always used because it results in smaller machine code. So, as it is now, on amd64 we produce slightly smaller boot binaries where size doesn't matter. Where size really matters (-Os) we produce identical binaries. If we decide that it makes sense to converge i386 and amd64 boot build options, which should we pick? [*] It is the current state of matter, but it is not necessary that it will always be the same. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Wed Oct 3 20:43:13 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DC3A4106564A; Wed, 3 Oct 2012 20:43:13 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id B2A948FC08; Wed, 3 Oct 2012 20:43:13 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 10BA7B91E; Wed, 3 Oct 2012 16:43:13 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Wed, 3 Oct 2012 15:12:37 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p20; KDE/4.5.5; amd64; ; ) References: <506C385C.3020400@FreeBSD.org> In-Reply-To: <506C385C.3020400@FreeBSD.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201210031512.37718.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 03 Oct 2012 16:43:13 -0400 (EDT) Cc: Andriy Gapon Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2012 20:43:14 -0000 On Wednesday, October 03, 2012 9:06:36 am Andriy Gapon wrote: > > Currently we produce "slightly" different binaries for x86 boot code depending > whether MACHINE_CPUARCH is i386 or amd64. I think that there is no good reason > for this, since in both cases we use exactly the same code and target the same > classes of machines. In other words, the binaries should be interchangeable[*]. > > The difference boils down to using -march=i386 on amd64 while i386 uses default > compiler flags, which are equivalent to -march=i486 -mtune=generic. > If my analysis is correct, the only thing affected by the flags in the boot code > is use of leave instruction when -Os is _not_ specified. > For -march=i386 our gcc prefers using leave. For -march=i486 it thinks that > movs+pops are faster than leave and so prefers to not use it. If -Os is > specified, then leave is always used because it results in smaller machine code. > > So, as it is now, on amd64 we produce slightly smaller boot binaries where size > doesn't matter. Where size really matters (-Os) we produce identical binaries. > > If we decide that it makes sense to converge i386 and amd64 boot build options, > which should we pick? > > [*] It is the current state of matter, but it is not necessary that it will always > be the same. Go for smaller binaries. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Oct 4 16:11:23 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 71256106566C; Thu, 4 Oct 2012 16:11:23 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) by mx1.freebsd.org (Postfix) with ESMTP id 3B3B78FC08; Thu, 4 Oct 2012 16:11:22 +0000 (UTC) Received: from [209.249.190.124] (port=50363 helo=gnnmac.hudson-trading.com) by vps.hungerhost.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.80) (envelope-from ) id 1TJo1C-0007o9-Br; Thu, 04 Oct 2012 12:11:22 -0400 Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) From: George Neville-Neil In-Reply-To: <201210021037.27762.jhb@freebsd.org> Date: Thu, 4 Oct 2012 12:11:21 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: <5BC1F38C-C354-4A01-B809-A8FE64824ABD@neville-neil.com> References: <201210020750.23358.jhb@freebsd.org> <201210021037.27762.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1498) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com Cc: Garrett Cooper , freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org, "Simon J. Gerraty" Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 16:11:23 -0000 On Oct 2, 2012, at 10:37 , John Baldwin wrote: > This is very non-obvious to the public at large (e.g. there was no = public=20 > response to one group's inquiry about the second ATF import for = example). =20 > Also, given that you had no idea that sgf@ and obrien@ were working on=20= > importing NetBSD's bmake as a prerequisite for ATF, it seems that = whatever=20 > discussions were held were not very detailed at best. I think it = would be=20 > good to have the various folks working on ATF to at least summarize = the=20 > current state of things and sketch out some sort of plan or roadmap = for future=20 > work in a public forum (such as atf@, though a summary mail would be = quite=20 > appropriate for arch@). I take partial responsibility for the privacy of the discussions = hitherto. My apologies, it should have be moved out onto a public list sooner. But, I would like to drive this to a solution on arch@. We don't have = an atf@, but we do have a test@ and testing@. We have too many mailing lists already, so let's finish this up here if we can and then=20 continue talking on testing@. Best, George From owner-freebsd-arch@FreeBSD.ORG Thu Oct 4 16:29:11 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C26A81065670; Thu, 4 Oct 2012 16:29:11 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (m209-73.dsl.rawbw.com [198.144.209.73]) by mx1.freebsd.org (Postfix) with ESMTP id 90E7A8FC16; Thu, 4 Oct 2012 16:29:11 +0000 (UTC) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.14.5/8.14.5) with ESMTP id q94GTAfW038242; Thu, 4 Oct 2012 09:29:10 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.14.5/8.14.5/Submit) id q94GTAFw038241; Thu, 4 Oct 2012 09:29:10 -0700 (PDT) (envelope-from david) Date: Thu, 4 Oct 2012 09:29:10 -0700 From: David Wolfskill To: George Neville-Neil Message-ID: <20121004162910.GQ23688@albert.catwhisker.org> References: <201210020750.23358.jhb@freebsd.org> <201210021037.27762.jhb@freebsd.org> <5BC1F38C-C354-4A01-B809-A8FE64824ABD@neville-neil.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="sxhug0Teuf3tiWmo" Content-Disposition: inline In-Reply-To: <5BC1F38C-C354-4A01-B809-A8FE64824ABD@neville-neil.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org, "Simon J. Gerraty" , freebsd-arch@freebsd.org Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 16:29:11 -0000 --sxhug0Teuf3tiWmo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 04, 2012 at 12:11:21PM -0400, George Neville-Neil wrote: > ... > But, I would like to drive this to a solution on arch@. We don't have an > atf@, but we do have a test@ and testing@. We have too many mailing > lists already, so let's finish this up here if we can and then=20 > continue talking on testing@. > ... Before folks get too excited about test@ & testing@: * test@ is for testing ability to send mail to FreeBSD.org (mailing lists). * testing@ *used* to exist, but was retired for lack of relevant traffic. * There's little difference in effort in resurrecting testing@ vs. creating atf@. Peace, david (current hat: part of postmaster@freebsd.org) --=20 David H. Wolfskill david@catwhisker.org Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --sxhug0Teuf3tiWmo Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlBtuVUACgkQmprOCmdXAD20EQCdH48KJDsJRSbTis+lKMJaMhKV +h8AniqLHtULN4kBAWZT2/nwSFe0/a5b =nuve -----END PGP SIGNATURE----- --sxhug0Teuf3tiWmo-- From owner-freebsd-arch@FreeBSD.ORG Thu Oct 4 16:42:36 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 14F80106564A; Thu, 4 Oct 2012 16:42:36 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9CE138FC14; Thu, 4 Oct 2012 16:42:35 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so968738oag.13 for ; Thu, 04 Oct 2012 09:42:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=cfhiCnX0umtxbR+tLgCvwAzPiDUiLQOAN+xvx1YK3so=; b=b+IlJ8vAbGB0G4QnzLsHs3lY9RPhDZYXLOKwOR8aLZMVjLUFF6RTohTfJxokaSr3Az jt99x8gyAvKdJQagpmCo69WsEvQjHTeuEVyX6yACyZX0k221+7HZlJk1peqCnJVLAMAH X6v/uUQbvqGTkjT/BGEFa9y5aQF2xMqjsAps3fgld3tRkB4C5Y2B7gyEcQUFOxuktWvm OwHKRtSnYsUkGPXayLT/RpHO+nGdJdMKzRg5PAoB5CbNNJgdyGTw+ClPdG//vBs5h7t0 8uRf9XO2gcSCvka902sRiP7l/TRxqODhOocGnV0kXsbDa0li9q82QdccHtn3PP0YCbSV Nv2g== MIME-Version: 1.0 Received: by 10.182.52.3 with SMTP id p3mr4726552obo.56.1349368949294; Thu, 04 Oct 2012 09:42:29 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Thu, 4 Oct 2012 09:42:29 -0700 (PDT) In-Reply-To: <201210021037.27762.jhb@freebsd.org> References: <201210020750.23358.jhb@freebsd.org> <201210021037.27762.jhb@freebsd.org> Date: Thu, 4 Oct 2012 09:42:29 -0700 Message-ID: From: Garrett Cooper To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, "Simon J. Gerraty" , freebsd-arch@freebsd.org Subject: Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 16:42:36 -0000 On Tue, Oct 2, 2012 at 7:37 AM, John Baldwin wrote: > On Tuesday, October 02, 2012 10:29:49 am Garrett Cooper wrote: >> On Tue, Oct 2, 2012 at 4:50 AM, John Baldwin wrote: >> >> ... >> >> > This sounds like a superior approach. It doesn't break any current use >> > cases while giving the ability to build multiple programs in the few >> > places that need it. It sounds like there are a few places under gnu/ >> > from Garrett's reply that might be able to make use of this as well. >> >> For the record, gnu/cc/cc_tools/Makefile is where I first spotted a >> potential "bsd.progs.mk" candidate. Most of the other code doesn't >> care given how things are organized in our source tree. >> >> > BTW, one general comment. There seem to be two completely independent >> > groups of folks working on ATF (e.g. there have been two different >> > imports of ATF into the tree in two different locations IIRC, and now >> > we have two different sets of patches to our system makefiles). >> > >> > Are these two groups talking to each other at all? I know in May that >> > many folks (certainly multiple vendors) are interested in ATF, and it >> > seems that both Juniper and Isilon have ported ATF internally. It seems >> > that it might be good for the two groups to work together to avoid >> > stomping on each other's toes. It seems there are some differences in >> > the two approaches that merit working out to avoid a lot of wasted >> > effort on both sides. >> >> Both parties (Isilon/Juniper) are converging on the ATF porting work >> that Giorgos/myself have done after talking at the FreeBSD Foundation >> meet-n-greet. I have contributed all of the patches that I have other >> to marcel for feedback. > > This is very non-obvious to the public at large (e.g. there was no public > response to one group's inquiry about the second ATF import for example). > Also, given that you had no idea that sgf@ and obrien@ were working on > importing NetBSD's bmake as a prerequisite for ATF, it seems that whatever > discussions were held were not very detailed at best. I think it would be > good to have the various folks working on ATF to at least summarize the > current state of things and sketch out some sort of plan or roadmap for future > work in a public forum (such as atf@, though a summary mail would be quite > appropriate for arch@). I'm in part to blame for this. There was some discussion -- but not at length; unfortunately no one from Juniper was present at the meet and greet; the information I got was second hand; I didn't follow up to figure out the exact details / clarify what I had in mind with the appropriate parties. That being said, I *sort* of understand what's going on now, although I'm still guessing as I haven't received any FreeBSD internal (developers@, etc) emails and all the discussion I have so far is between gnn@, marcel@, gibbs@, mlaier@, and mdf@. For all intents and purposes I've been paused for a few weeks because of other things, so no harm, no foul, but I'd like to know what all is being contributed back from Juniper in terms of tests, ATF integration into the build system (which I've given back to marcel@/gnn@, but haven't received feedback for yet -- probably because they're busy), etc so I can better avoid duplicated effort and help the cause of creating a maintainable/testable FreeBSD. As far as what Isilon is contributing back, I wouldn't look any further than my perforce repo; the original goal as of last BSDCan was to contribute back `isi_check` (custom wrapper around GNU libcheck), and maybe some of our tests are written for isi_check: the problem with that plan is that it depends on GNU [lib]check -- yet another test infrastructure -- diverges us more unnecessarily from NetBSD (and there are several things we want to grab from NetBSD and contribute back instead of forging ahead down our own path), the tests would need to be audited and cleaned up to use generic macros and check for generic functionality, etc. Also, it would help to have generic system tests similar to LTP's breadth in the tree (so that way we can avoid breakage before things are committed to current). There are some functional gaps that I like to fill in with ATF that GNU libcheck does well [with fixtures] and there are some inconsistencies between the ATF C and C++ bindings I'm working on enhancing More details about what I planned on doing can be found here: http://wiki.freebsd.org/TestingFreeBSD -- although it deserves updating when looking at the structure and dealing with the "patches" (I need to update it to "just work" with the latest current). Thanks! -Garrett From owner-freebsd-arch@FreeBSD.ORG Thu Oct 4 16:44:31 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2DC671065673; Thu, 4 Oct 2012 16:44:31 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id AB1BE8FC1F; Thu, 4 Oct 2012 16:44:29 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so971775oag.13 for ; Thu, 04 Oct 2012 09:44:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=OIcPlwJWYOyaCDKC5Jp/d/+vu8n+MMMOb9pLTG+txqg=; b=Fg+8tf+ks2qTkyrHzMLDFjur6EFrOQE7PwJvfK5zV/lsnEmKSkgS7AFj/W6LQI0r4F ja/XyKBYTbUdEFsc/H+R4Bl2v/kio/RyCI9HD8xfWc/n2+oz+87VySXO9oyDH92qHrTa qpfKi/bN0jtF2CJPDEDN8A/n1/1Cdp+gwQBebiApSUClEzBrpSL7CnkFI6pCEgdKsNLX pog19tIkktDhwkmHdegg+YllU2AeNEdDpbZSLxfDo+2HCK8/XkftOdUKXC3ykqF4M4Aq qidjHDEEQKkRyiK3rrOEQk09E8++1O5b9aoZlrQlMfpMeYTwEBq/EkqInANL4Xozx/1t SyLA== MIME-Version: 1.0 Received: by 10.182.10.6 with SMTP id e6mr4831252obb.16.1349369069022; Thu, 04 Oct 2012 09:44:29 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Thu, 4 Oct 2012 09:44:28 -0700 (PDT) In-Reply-To: <20121004162910.GQ23688@albert.catwhisker.org> References: <201210020750.23358.jhb@freebsd.org> <201210021037.27762.jhb@freebsd.org> <5BC1F38C-C354-4A01-B809-A8FE64824ABD@neville-neil.com> <20121004162910.GQ23688@albert.catwhisker.org> Date: Thu, 4 Oct 2012 09:44:28 -0700 Message-ID: From: Garrett Cooper To: David Wolfskill Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, George Neville-Neil , freebsd-arch@freebsd.org, "Simon J. Gerraty" Subject: Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 16:44:31 -0000 On Thu, Oct 4, 2012 at 9:29 AM, David Wolfskill wrote: > On Thu, Oct 04, 2012 at 12:11:21PM -0400, George Neville-Neil wrote: >> ... >> But, I would like to drive this to a solution on arch@. We don't have an >> atf@, but we do have a test@ and testing@. We have too many mailing >> lists already, so let's finish this up here if we can and then >> continue talking on testing@. >> ... > > Before folks get too excited about test@ & testing@: > > * test@ is for testing ability to send mail to FreeBSD.org (mailing > lists). > > * testing@ *used* to exist, but was retired for lack of relevant > traffic. > > * There's little difference in effort in resurrecting testing@ vs. > creating atf@. I think that resurrecting testing@ makes more sense as creating an atf specific list seems a bit too focused on ATF, primarily because atf is being "partially superseded" by kyua (pronounced Q-A) in the near future. Thanks! -Garrett From owner-freebsd-arch@FreeBSD.ORG Thu Oct 4 17:24:06 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 54B15106564A; Thu, 4 Oct 2012 17:24:06 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from exprod7og119.obsmtp.com (exprod7og119.obsmtp.com [64.18.2.16]) by mx1.freebsd.org (Postfix) with ESMTP id ADE8E8FC0A; Thu, 4 Oct 2012 17:24:04 +0000 (UTC) Received: from P-EMHUB03-HQ.jnpr.net ([66.129.224.36]) (using TLSv1) by exprod7ob119.postini.com ([64.18.6.12]) with SMTP ID DSNKUG3GMxBnovHKF+pTeUy1ZIIkq/8e3SrP@postini.com; Thu, 04 Oct 2012 10:24:05 PDT Received: from magenta.juniper.net (172.17.27.123) by P-EMHUB03-HQ.jnpr.net (172.24.192.33) with Microsoft SMTP Server (TLS) id 8.3.213.0; Thu, 4 Oct 2012 10:22:05 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id q94HM4h44676; Thu, 4 Oct 2012 10:22:04 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 0518958093; Thu, 4 Oct 2012 10:22:04 -0700 (PDT) To: Garrett Cooper In-Reply-To: References: <201210020750.23358.jhb@freebsd.org> <201210021037.27762.jhb@freebsd.org> Comments: In-reply-to: Garrett Cooper message dated "Thu, 04 Oct 2012 09:42:29 -0700." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Thu, 4 Oct 2012 10:22:03 -0700 Message-ID: <20121004172204.0518958093@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain Cc: freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Fwd: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 17:24:06 -0000 On Thu, 4 Oct 2012 09:42:29 -0700, Garrett Cooper writes: >I'd like to know what all is >being contributed back from Juniper in terms of tests, ATF integration >into the build system (which I've given back to marcel@/gnn@, but We aim to contribute build improvments, and integration of ATF was part of that. We had planned on doing the ATF import etc, but didn't want to re-invent wheels by trying to make it work without bmake. Or making a dog's breakfast out of bsd.*.mk Speaking of which, the initial commit (which should happen "real soon now" ;-) is a minimal set of changes to allow buildworld etc using bmake and allow folk who want to, to install bmake as make - as discussed at the last devsummit in Cambridge. Anyway, back to ATF, as mentioned earlier in this thread, I use atf.test.mk in our build rather than netbsd's bsd.test.mk, and we put all the test makefiles in a tests/ subdir of the lib or prog in question. This has important ramifications when it comes to wanting to build the tree in meta mode and have unit-tests build and run as an integral part of the build (or at least the option of doing that). As for meta mode, there is a projects/bmake branch which is a bit out of sync with head, but which I think has meta mode enabled (my internal mirror of it does ;-). It isn't ready for prime-time yet, a lot of the stuff in local.sys.mk there needs to migrate to sys.mk or similar, but that should probably wait until bmake is the default make, and there is also the need for more discussion here. But with a couple of env variables set, people should be able to play with it, to see what we are talking about. The next steps will focus on being able to have bmake the default - which means dealing with ports. I've a "patch", that's air-quotes, because I don't think a patch will suffice for a large moving target, rather its a script to run against a ports tree. Once the ports folk are happy that a bmake flavored ports tree can be built and used ok on an older base system, the final cutover can be planned. Hope that helps --sjg From owner-freebsd-arch@FreeBSD.ORG Thu Oct 4 20:02:26 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5A416106566C; Thu, 4 Oct 2012 20:02:26 +0000 (UTC) (envelope-from dimitry@andric.com) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) by mx1.freebsd.org (Postfix) with ESMTP id 13B218FC16; Thu, 4 Oct 2012 20:02:25 +0000 (UTC) Received: from [192.168.0.6] (spaceball.home.andric.com [192.168.0.6]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 898695C59; Thu, 4 Oct 2012 22:02:19 +0200 (CEST) Message-ID: <506DEB4C.5020508@andric.com> Date: Thu, 04 Oct 2012 22:02:20 +0200 From: Dimitry Andric User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121002 Thunderbird/16.0 MIME-Version: 1.0 To: Andriy Gapon References: <506C385C.3020400@FreeBSD.org> In-Reply-To: <506C385C.3020400@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 20:02:26 -0000 On 2012-10-03 15:06, Andriy Gapon wrote: > Currently we produce "slightly" different binaries for x86 boot code depending > whether MACHINE_CPUARCH is i386 or amd64. I think that there is no good reason > for this, since in both cases we use exactly the same code and target the same > classes of machines. In other words, the binaries should be interchangeable[*]. > > The difference boils down to using -march=i386 on amd64 while i386 uses default > compiler flags, which are equivalent to -march=i486 -mtune=generic. Yes, I also noticed this inconsistency during some other work in sys/boot, and I ended up with this diff in my backlog: Index: sys/boot/i386/Makefile.inc =================================================================== --- sys/boot/i386/Makefile.inc (revision 241194) +++ sys/boot/i386/Makefile.inc (working copy) @@ -5,12 +5,13 @@ BINDIR?= /boot LOADER_ADDRESS?=0x200000 -CFLAGS+= -ffreestanding -mpreferred-stack-boundary=2 \ +CFLAGS+= -march=i386 -ffreestanding -mpreferred-stack-boundary=2 \ -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -msoft-float +NO_CPU_CFLAGS= LDFLAGS+= -nostdlib .if ${MACHINE_CPUARCH} == "amd64" -CFLAGS+= -m32 -march=i386 +CFLAGS+= -m32 ACFLAGS+= -m32 LDFLAGS+= -m elf_i386_fbsd AFLAGS+= --32 > If my analysis is correct, the only thing affected by the flags in the boot code > is use of leave instruction when -Os is _not_ specified. > For -march=i386 our gcc prefers using leave. For -march=i486 it thinks that > movs+pops are faster than leave and so prefers to not use it. If -Os is > specified, then leave is always used because it results in smaller machine code. > > So, as it is now, on amd64 we produce slightly smaller boot binaries where size > doesn't matter. Where size really matters (-Os) we produce identical binaries. > > If we decide that it makes sense to converge i386 and amd64 boot build options, > which should we pick? Well, do we still officially support any real i386 machines? If so, we should still use -march=i386 for the boot code. Otherwise, let's start using -march=i486 explicitly. So like: Index: sys/boot/i386/Makefile.inc =================================================================== --- sys/boot/i386/Makefile.inc (revision 241194) +++ sys/boot/i386/Makefile.inc (working copy) @@ -5,12 +5,13 @@ BINDIR?= /boot LOADER_ADDRESS?=0x200000 -CFLAGS+= -ffreestanding -mpreferred-stack-boundary=2 \ +CFLAGS+= -march=i486 -ffreestanding -mpreferred-stack-boundary=2 \ -mno-mmx -mno-3dnow -mno-sse -mno-sse2 -mno-sse3 -msoft-float +NO_CPU_CFLAGS= LDFLAGS+= -nostdlib .if ${MACHINE_CPUARCH} == "amd64" -CFLAGS+= -m32 -march=i386 +CFLAGS+= -m32 ACFLAGS+= -m32 LDFLAGS+= -m elf_i386_fbsd AFLAGS+= --32 From owner-freebsd-arch@FreeBSD.ORG Thu Oct 4 21:42:38 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1DF7B106566B; Thu, 4 Oct 2012 21:42:38 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id D3C4F8FC0A; Thu, 4 Oct 2012 21:42:37 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id C78166476; Thu, 4 Oct 2012 23:42:35 +0200 (CEST) Received: by ds4.des.no (Postfix, from userid 1001) id 858238092; Thu, 4 Oct 2012 23:42:35 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Dimitry Andric References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> Date: Thu, 04 Oct 2012 23:42:35 +0200 In-Reply-To: <506DEB4C.5020508@andric.com> (Dimitry Andric's message of "Thu, 04 Oct 2012 22:02:20 +0200") Message-ID: <86haq9hq2c.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Andriy Gapon , freebsd-arch@FreeBSD.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Oct 2012 21:42:38 -0000 Dimitry Andric writes: > Well, do we still officially support any real i386 machines? No, 486 and up only. Personally, I think we should ship 586 binaries (pentium-mmx) by default. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 03:32:57 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 919AC106566B; Fri, 5 Oct 2012 03:32:57 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 0BFB08FC0A; Fri, 5 Oct 2012 03:32:55 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q953WuK2052369; Fri, 5 Oct 2012 06:32:56 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q953Wie8074074; Fri, 5 Oct 2012 06:32:44 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q953Wiwa074073; Fri, 5 Oct 2012 06:32:44 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 5 Oct 2012 06:32:44 +0300 From: Konstantin Belousov To: Dag-Erling Sm??rgrav Message-ID: <20121005033244.GL35915@deviant.kiev.zoral.com.ua> References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <86haq9hq2c.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="i8OEs6gA6Pkg4k/n" Content-Disposition: inline In-Reply-To: <86haq9hq2c.fsf@ds4.des.no> User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Dimitry Andric , Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 03:32:57 -0000 --i8OEs6gA6Pkg4k/n Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 04, 2012 at 11:42:35PM +0200, Dag-Erling Sm??rgrav wrote: > Dimitry Andric writes: > > Well, do we still officially support any real i386 machines? >=20 > No, 486 and up only. Personally, I think we should ship 586 binaries > (pentium-mmx) by default. There is absolutely no architectural difference between usermode ISA between i386 and pentiums, ignoring SMP-support instructions, which are usually not emited by the compiler anyway. Really interesting stuff started appearing with pentium pro, like CMOV instructions. Even more important, -march=3Dpentiumpro generates much better -fPIC code (probably could be activated by -mcpu=3Dpentiumpro). Anyway, we shall support 486 in the default 32bit build. --i8OEs6gA6Pkg4k/n Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlBuVNwACgkQC3+MBN1Mb4hKZQCfZ5AkIUMCYn+F7ajxjfrSSr0B SoEAoMqj+uGFrIw7fYRc5jg+G/hEHcTT =OURS -----END PGP SIGNATURE----- --i8OEs6gA6Pkg4k/n-- From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 05:15:18 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 77A3B1065670 for ; Fri, 5 Oct 2012 05:15:18 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 869A48FC08 for ; Fri, 5 Oct 2012 05:15:17 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id IAA26090; Fri, 05 Oct 2012 08:15:08 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1TK0Ff-000OcS-NX; Fri, 05 Oct 2012 08:15:07 +0300 Message-ID: <506E6CDA.4080507@FreeBSD.org> Date: Fri, 05 Oct 2012 08:15:06 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120913 Thunderbird/15.0.1 MIME-Version: 1.0 To: Dimitry Andric References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> In-Reply-To: <506DEB4C.5020508@andric.com> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 05:15:18 -0000 on 04/10/2012 23:02 Dimitry Andric said the following: > Well, do we still officially support any real i386 machines? If so, we > should still use -march=i386 for the boot code. Otherwise, let's start > using -march=i486 explicitly. As I mentioned earlier, the only difference for boot code is use of 'leave' instruction. I don't think -march=i486 buys us much, if anything, except for "coolness factor" (i486 is "cooler" than i386). On the other hand it makes binaries larger. So... -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 07:08:38 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 12CFC106566C; Fri, 5 Oct 2012 07:08:38 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id C5A4F8FC14; Fri, 5 Oct 2012 07:08:37 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 7D4C065BF; Fri, 5 Oct 2012 09:08:36 +0200 (CEST) Received: by ds4.des.no (Postfix, from userid 1001) id 4BDF78123; Fri, 5 Oct 2012 09:08:36 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Konstantin Belousov References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <86haq9hq2c.fsf@ds4.des.no> <20121005033244.GL35915@deviant.kiev.zoral.com.ua> Date: Fri, 05 Oct 2012 09:08:35 +0200 In-Reply-To: <20121005033244.GL35915@deviant.kiev.zoral.com.ua> (Konstantin Belousov's message of "Fri, 5 Oct 2012 06:32:44 +0300") Message-ID: <86y5jll7kc.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Dimitry Andric , Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 07:08:38 -0000 Konstantin Belousov writes: > Dag-Erling Sm=C3=B8rgrav writes: > > Dimitry Andric writes: > > > Well, do we still officially support any real i386 machines? > > No, 486 and up only. Personally, I think we should ship 586 > > binaries (pentium-mmx) by default. > There is absolutely no architectural difference between usermode ISA > between i386 and pentiums, ignoring SMP-support instructions, which > are usually not emited by the compiler anyway. By "binaries" I mean ISOs and freebsd-update, including the kernel. (actually, it's the kernel I care the most about) > Really interesting stuff started appearing with pentium pro, like CMOV > instructions. Even more important, -march=3Dpentiumpro generates much > better -fPIC code (probably could be activated by -mcpu=3Dpentiumpro). Which is why most Linux distributions target 686, but we can't if we want to support small systems like the AMD Geode-based soekris net4xxx and net5xxx out of the box. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 07:40:53 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 481791065670; Fri, 5 Oct 2012 07:40:53 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id E62B78FC0A; Fri, 5 Oct 2012 07:40:52 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so1938370oag.13 for ; Fri, 05 Oct 2012 00:40:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=Y8nqbCv8dfPdeBVg8ePO/gwD4zl+YQd+i3NOQDVYKfU=; b=vm4y87dygNNFZDXsYMZDMkkHHkXZPNqCUgkshVpSW0U1db2teLHhpwA7JIr3Njz1iQ sOH9Yr1eI5DjPDZlE0t9raavvoiDkNYz97aI5OkgAqF7F9TVUaiD1WXRccQ4/nB3lbYN 3il3ykbmx7gMepKkRvO3NfyEHSIWWf6gJZRzuvFqlMt/sN/scrHwS+txx7k6cjAOpCx0 d2cJiO4a30AhAsFvyyGphQTRnstop1r0jtQ3DNy7zyP04dt1vvz++7oczcUzAxZUZMLv abAmyPrpeAosBG9mZZzlEUsWqOcELI4j9ANtia2M8bv2ppvt52QBErAVkgFcj833aGrF 1JaA== MIME-Version: 1.0 Received: by 10.182.218.37 with SMTP id pd5mr6469060obc.24.1349422852035; Fri, 05 Oct 2012 00:40:52 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Fri, 5 Oct 2012 00:40:51 -0700 (PDT) In-Reply-To: <86y5jll7kc.fsf@ds4.des.no> References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <86haq9hq2c.fsf@ds4.des.no> <20121005033244.GL35915@deviant.kiev.zoral.com.ua> <86y5jll7kc.fsf@ds4.des.no> Date: Fri, 5 Oct 2012 00:40:51 -0700 Message-ID: From: Garrett Cooper To: =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Konstantin Belousov , Dimitry Andric , Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 07:40:53 -0000 On Fri, Oct 5, 2012 at 12:08 AM, Dag-Erling Sm=F8rgrav wrote: > Konstantin Belousov writes: >> Dag-Erling Sm=F8rgrav writes: >> > Dimitry Andric writes: >> > > Well, do we still officially support any real i386 machines? >> > No, 486 and up only. Personally, I think we should ship 586 >> > binaries (pentium-mmx) by default. >> There is absolutely no architectural difference between usermode ISA >> between i386 and pentiums, ignoring SMP-support instructions, which >> are usually not emited by the compiler anyway. > > By "binaries" I mean ISOs and freebsd-update, including the kernel. > > (actually, it's the kernel I care the most about) > >> Really interesting stuff started appearing with pentium pro, like CMOV >> instructions. Even more important, -march=3Dpentiumpro generates much >> better -fPIC code (probably could be activated by -mcpu=3Dpentiumpro). > > Which is why most Linux distributions target 686, but we can't if we > want to support small systems like the AMD Geode-based soekris net4xxx > and net5xxx out of the box. I would target the appropriate architecture (amd64) where it matters (amd64), and target the lowest sane common denominator on i386. In reality, what does a couple MB mean on amd64 vs i386? Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 07:43:09 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id ECC1E1065672; Fri, 5 Oct 2012 07:43:09 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id 949038FC17; Fri, 5 Oct 2012 07:43:09 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id n9so1940340oag.13 for ; Fri, 05 Oct 2012 00:43:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=IFBcjXaXXCd/3p+N2ICa/CEzzclgTzBN2LCt7F/uIOA=; b=ctcWxrswuHQrF+BkUUkABoFBaYtwJmcpTeQ81oY1HhrojrHea3Bq4ZcFwd4N2AOkTP 53msGgef/CsU0oSgoVSpALPmMK1TxJbxmIHqj0GSq2u0AhDNZRjdEgHP2UItlV+TJTKZ EO+j2R8h9c6M1d4oqLUQlXjuFQ3DN/ZRJqA5mKBcA2kBZKa61xirvhYdPLEXzMDo9gqD qS84HETfJYms+3Qayd9XxzDOI/R9fAJZ2Q0mVsulhqTLoV+X6c7uz5SJuaVGospLAbmg 3OTnsgpVXV7yrqkpNjrieYW0MeX3oK47/VMBrzjV2MxtAFLd3D9hGt3mFWWeiFF7O/DA 9bAQ== MIME-Version: 1.0 Received: by 10.60.11.162 with SMTP id r2mr6298005oeb.114.1349422989028; Fri, 05 Oct 2012 00:43:09 -0700 (PDT) Received: by 10.76.142.201 with HTTP; Fri, 5 Oct 2012 00:43:08 -0700 (PDT) In-Reply-To: References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <86haq9hq2c.fsf@ds4.des.no> <20121005033244.GL35915@deviant.kiev.zoral.com.ua> <86y5jll7kc.fsf@ds4.des.no> Date: Fri, 5 Oct 2012 00:43:08 -0700 Message-ID: From: Garrett Cooper To: =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Konstantin Belousov , Dimitry Andric , Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 07:43:10 -0000 On Fri, Oct 5, 2012 at 12:40 AM, Garrett Cooper wrote: > On Fri, Oct 5, 2012 at 12:08 AM, Dag-Erling Sm=F8rgrav wrote= : >> Konstantin Belousov writes: >>> Dag-Erling Sm=F8rgrav writes: >>> > Dimitry Andric writes: >>> > > Well, do we still officially support any real i386 machines? >>> > No, 486 and up only. Personally, I think we should ship 586 >>> > binaries (pentium-mmx) by default. >>> There is absolutely no architectural difference between usermode ISA >>> between i386 and pentiums, ignoring SMP-support instructions, which >>> are usually not emited by the compiler anyway. >> >> By "binaries" I mean ISOs and freebsd-update, including the kernel. >> >> (actually, it's the kernel I care the most about) >> >>> Really interesting stuff started appearing with pentium pro, like CMOV >>> instructions. Even more important, -march=3Dpentiumpro generates much >>> better -fPIC code (probably could be activated by -mcpu=3Dpentiumpro). >> >> Which is why most Linux distributions target 686, but we can't if we >> want to support small systems like the AMD Geode-based soekris net4xxx >> and net5xxx out of the box. > > I would target the appropriate architecture (amd64) where it > matters (amd64), and target the lowest sane common denominator on > i386. In reality, what does a couple MB mean on amd64 vs i386? To clarify... s/couple MB/couple MB in the loader program/ Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 08:04:28 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 508401065670; Fri, 5 Oct 2012 08:04:28 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 0D5F08FC14; Fri, 5 Oct 2012 08:04:27 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 50BE265D6; Fri, 5 Oct 2012 10:04:27 +0200 (CEST) Received: by ds4.des.no (Postfix, from userid 1001) id 26358812D; Fri, 5 Oct 2012 10:04:27 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Garrett Cooper References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <86haq9hq2c.fsf@ds4.des.no> <20121005033244.GL35915@deviant.kiev.zoral.com.ua> <86y5jll7kc.fsf@ds4.des.no> Date: Fri, 05 Oct 2012 10:04:26 +0200 In-Reply-To: (Garrett Cooper's message of "Fri, 5 Oct 2012 00:40:51 -0700") Message-ID: <86txu9l4z9.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Konstantin Belousov , Dimitry Andric , Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 08:04:28 -0000 Garrett Cooper writes: > I would target the appropriate architecture (amd64) where it matters > (amd64), and target the lowest sane common denominator on i386. In > reality, what does a couple MB mean on amd64 vs i386? 1) Nobody mentioned amd64 - this is about i386. 2) It's not a question of *size* but of *performance*. By targeting the least capable platform that our users are likely to encounter (pentium-mmx) instead of one which is virtually eradicated (486), we can use features that are available on the former but not the latter. Someone said they'd like to target SSE2, but that would leave many common embedded systems out in the cold. If we do that, we should provide two sets of binaries; one set for sse2-capable machines (which covers all i386 desktop and server machines made in the last ten years) and one set for pentium-mmx (which covers the soekris and other popular SFF / embedded systems). DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 08:05:21 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EBF6A106564A; Fri, 5 Oct 2012 08:05:21 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id A8B458FC1C; Fri, 5 Oct 2012 08:05:21 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 1997965D8; Fri, 5 Oct 2012 10:05:21 +0200 (CEST) Received: by ds4.des.no (Postfix, from userid 1001) id E9B7C812F; Fri, 5 Oct 2012 10:05:20 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Garrett Cooper References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <86haq9hq2c.fsf@ds4.des.no> <20121005033244.GL35915@deviant.kiev.zoral.com.ua> <86y5jll7kc.fsf@ds4.des.no> Date: Fri, 05 Oct 2012 10:05:20 +0200 In-Reply-To: (Garrett Cooper's message of "Fri, 5 Oct 2012 00:43:08 -0700") Message-ID: <86pq4xl4xr.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Konstantin Belousov , Dimitry Andric , Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 08:05:22 -0000 Garrett Cooper writes: > To clarify... > > s/couple MB/couple MB in the loader program/ The loader is unimportant. It is only used at boot. What's important is the kernel. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 08:25:14 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DB0DB106566C; Fri, 5 Oct 2012 08:25:14 +0000 (UTC) (envelope-from dimitry@andric.com) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) by mx1.freebsd.org (Postfix) with ESMTP id 96DE38FC1D; Fri, 5 Oct 2012 08:25:14 +0000 (UTC) Received: from [192.168.0.6] (spaceball.home.andric.com [192.168.0.6]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 42FC35C59; Fri, 5 Oct 2012 10:25:13 +0200 (CEST) Message-ID: <506E996C.60203@andric.com> Date: Fri, 05 Oct 2012 10:25:16 +0200 From: Dimitry Andric User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121002 Thunderbird/16.0 MIME-Version: 1.0 To: Andriy Gapon References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <506E6CDA.4080507@FreeBSD.org> In-Reply-To: <506E6CDA.4080507@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 08:25:15 -0000 On 2012-10-05 07:15, Andriy Gapon wrote: > on 04/10/2012 23:02 Dimitry Andric said the following: >> Well, do we still officially support any real i386 machines? If so, we >> should still use -march=i386 for the boot code. Otherwise, let's start >> using -march=i486 explicitly. > As I mentioned earlier, the only difference for boot code is use of 'leave' > instruction. I don't think -march=i486 buys us much, if anything, except for > "coolness factor" (i486 is "cooler" than i386). On the other hand it makes > binaries larger. So... Yes, the boot loader is a special case anyway. If -march=i386 makes the binary just a little bit smaller, let's use that. At least then the used flags will be consistent across the i386 and amd64 builds. From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 11:23:31 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5773B106574F; Fri, 5 Oct 2012 11:23:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id A96DE8FC14; Fri, 5 Oct 2012 11:23:30 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q95BNPB6001609; Fri, 5 Oct 2012 14:23:25 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q95BNDcl075848; Fri, 5 Oct 2012 14:23:13 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q95BND98075847; Fri, 5 Oct 2012 14:23:13 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 5 Oct 2012 14:23:13 +0300 From: Konstantin Belousov To: Dag-Erling Sm??rgrav Message-ID: <20121005112313.GN35915@deviant.kiev.zoral.com.ua> References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <86haq9hq2c.fsf@ds4.des.no> <20121005033244.GL35915@deviant.kiev.zoral.com.ua> <86y5jll7kc.fsf@ds4.des.no> <86txu9l4z9.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="EM2WAHSp0JAC5kZ2" Content-Disposition: inline In-Reply-To: <86txu9l4z9.fsf@ds4.des.no> User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Garrett Cooper , Dimitry Andric , Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 11:23:31 -0000 --EM2WAHSp0JAC5kZ2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Oct 05, 2012 at 10:04:26AM +0200, Dag-Erling Sm??rgrav wrote: > Garrett Cooper writes: > > I would target the appropriate architecture (amd64) where it matters > > (amd64), and target the lowest sane common denominator on i386. In > > reality, what does a couple MB mean on amd64 vs i386? >=20 > 1) Nobody mentioned amd64 - this is about i386. >=20 > 2) It's not a question of *size* but of *performance*. By targeting the > least capable platform that our users are likely to encounter > (pentium-mmx) instead of one which is virtually eradicated (486), we > can use features that are available on the former but not the latter. So what ISA additions do you expect to get advantage of by switching to pentium-mmx from 486 ? As I already said, I am not aware of any. I can only think of cmpxchg8b, which would eventually allow to implement 64bit atomics on i386, but this has nothing to do with the compiler. >=20 > Someone said they'd like to target SSE2, but that would leave many > common embedded systems out in the cold. If we do that, we should > provide two sets of binaries; one set for sse2-capable machines > (which covers all i386 desktop and server machines made in the last > ten years) and one set for pentium-mmx (which covers the soekris and > other popular SFF / embedded systems). >=20 > DES > --=20 > Dag-Erling Sm??rgrav - des@des.no --EM2WAHSp0JAC5kZ2 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlBuwyEACgkQC3+MBN1Mb4jzGwCg3uct1G5C17tJW0mU8Du9PhQ2 7osAoPAqWb6iSzXLJL+Ytd+WNjxchcVm =nVAM -----END PGP SIGNATURE----- --EM2WAHSp0JAC5kZ2-- From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 13:22:35 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E17B61065740; Fri, 5 Oct 2012 13:22:34 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 9C92C8FC16; Fri, 5 Oct 2012 13:22:33 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp.des.no (Postfix) with ESMTP id 2AF1D665F; Fri, 5 Oct 2012 15:22:32 +0200 (CEST) Received: by ds4.des.no (Postfix, from userid 1001) id D77678167; Fri, 5 Oct 2012 15:22:31 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Konstantin Belousov References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <86haq9hq2c.fsf@ds4.des.no> <20121005033244.GL35915@deviant.kiev.zoral.com.ua> <86y5jll7kc.fsf@ds4.des.no> <86txu9l4z9.fsf@ds4.des.no> <20121005112313.GN35915@deviant.kiev.zoral.com.ua> Date: Fri, 05 Oct 2012 15:22:31 +0200 In-Reply-To: <20121005112313.GN35915@deviant.kiev.zoral.com.ua> (Konstantin Belousov's message of "Fri, 5 Oct 2012 14:23:13 +0300") Message-ID: <86a9w1kq94.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Garrett Cooper , Dimitry Andric , Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 13:22:35 -0000 Konstantin Belousov writes: > So what ISA additions do you expect to get advantage of by switching > to pentium-mmx from 486 ? As I already said, I am not aware of any. The TSC, for one. MMX, and the ability to use MMX registers to copy data. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 13:36:38 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D5A801065672; Fri, 5 Oct 2012 13:36:38 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 335C98FC0A; Fri, 5 Oct 2012 13:36:37 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q95DaSNF012975; Fri, 5 Oct 2012 16:36:28 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q95DaG9F076595; Fri, 5 Oct 2012 16:36:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q95DaG5t076594; Fri, 5 Oct 2012 16:36:16 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 5 Oct 2012 16:36:16 +0300 From: Konstantin Belousov To: Dag-Erling Sm??rgrav Message-ID: <20121005133616.GP35915@deviant.kiev.zoral.com.ua> References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <86haq9hq2c.fsf@ds4.des.no> <20121005033244.GL35915@deviant.kiev.zoral.com.ua> <86y5jll7kc.fsf@ds4.des.no> <86txu9l4z9.fsf@ds4.des.no> <20121005112313.GN35915@deviant.kiev.zoral.com.ua> <86a9w1kq94.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="aTq0RFdLe0rSEdju" Content-Disposition: inline In-Reply-To: <86a9w1kq94.fsf@ds4.des.no> User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Garrett Cooper , Dimitry Andric , Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 13:36:38 -0000 --aTq0RFdLe0rSEdju Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Oct 05, 2012 at 03:22:31PM +0200, Dag-Erling Sm??rgrav wrote: > Konstantin Belousov writes: > > So what ISA additions do you expect to get advantage of by switching > > to pentium-mmx from 486 ? As I already said, I am not aware of any. >=20 > The TSC, for one. MMX, and the ability to use MMX registers to copy > data. TSC is used regardless of the compiler flags, we use it if CPU claims that TSC is supported, even in usermode. Compiler never generates MMX copies. More, in kernel, the manual FPU context save/restore is needed around the FPU/MMX register file access. --aTq0RFdLe0rSEdju Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlBu4k8ACgkQC3+MBN1Mb4i/WQCfc3ri41Dl3IEnbO7hI+29sdci 8+AAoKghSanVzrgfGt/OVQpzU1vrWyng =wd3Q -----END PGP SIGNATURE----- --aTq0RFdLe0rSEdju-- From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 15:54:48 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 77C06106566B; Fri, 5 Oct 2012 15:54:48 +0000 (UTC) (envelope-from uqs@FreeBSD.org) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 061928FC0A; Fri, 5 Oct 2012 15:54:47 +0000 (UTC) Received: from localhost (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by acme.spoerlein.net (8.14.5/8.14.5) with ESMTP id q95FsfgR072978 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 5 Oct 2012 17:54:41 +0200 (CEST) (envelope-from uqs@FreeBSD.org) Date: Fri, 5 Oct 2012 17:54:41 +0200 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Dag-Erling =?utf-8?B?U23DuHJncmF2?= Message-ID: <20121005155441.GL69724@acme.spoerlein.net> Mail-Followup-To: Dag-Erling =?utf-8?B?U23DuHJncmF2?= , Garrett Cooper , Konstantin Belousov , Dimitry Andric , Andriy Gapon , freebsd-arch@freebsd.org References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <86haq9hq2c.fsf@ds4.des.no> <20121005033244.GL35915@deviant.kiev.zoral.com.ua> <86y5jll7kc.fsf@ds4.des.no> <86txu9l4z9.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <86txu9l4z9.fsf@ds4.des.no> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Garrett Cooper , Konstantin Belousov , Dimitry Andric , Andriy Gapon , freebsd-arch@FreeBSD.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 15:54:48 -0000 On Fri, 2012-10-05 at 10:04:26 +0200, Dag-Erling Smørgrav wrote: > Garrett Cooper writes: > > I would target the appropriate architecture (amd64) where it matters > > (amd64), and target the lowest sane common denominator on i386. In > > reality, what does a couple MB mean on amd64 vs i386? > > 1) Nobody mentioned amd64 - this is about i386. > > 2) It's not a question of *size* but of *performance*. By targeting the > least capable platform that our users are likely to encounter > (pentium-mmx) instead of one which is virtually eradicated (486), we > can use features that are available on the former but not the latter. > > Someone said they'd like to target SSE2, but that would leave many > common embedded systems out in the cold. If we do that, we should > provide two sets of binaries; one set for sse2-capable machines > (which covers all i386 desktop and server machines made in the last > ten years) and one set for pentium-mmx (which covers the soekris and > other popular SFF / embedded systems). Seriously? How about we leave i386 as is for the embedded folks and just move forward with amd64. Any improvements made on our i386 support are useless to about 98% of our users. It's 2012 people ... kthxbai Uli From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 16:12:44 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7F625106566B for ; Fri, 5 Oct 2012 16:12:44 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id DC8C08FC08 for ; Fri, 5 Oct 2012 16:12:43 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA00795 for ; Fri, 05 Oct 2012 19:12:42 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <506F06FA.4050804@FreeBSD.org> Date: Fri, 05 Oct 2012 19:12:42 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120911 Thunderbird/15.0.1 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: drivers for desktop hardware monitoring chips X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 16:12:44 -0000 Currently FreeBSD is severely lacking in support for basic hardware monitoring on desktop-class systems. Such systems typically do not have IPMI or other similar types of agents. Instead they usually have a hardware monitoring function in a Super I/O chip or less frequently in a dedicated chip. Popular manufacturers of such chips are Nuvoton (formerly Winbond) and ITE. FreeBSD doesn't provide any support for this hardware neither in kernel nor in base system userland tools. There are a number of tools in ports (e.g. mbmon), but they are of varying quality and most of them are very out-of-date. So they do not properly support recent (and no so recent) hardware. The userland tools are dying out mostly because most of other popular operating systems have built-in support for such hardware and thus there is no incentive to support any cross-platform userland utilities. Which is not trivial too, because of different policies and interfaces to access system/hardware resources. So I think that it is time that we also provide some support for this kind of hardware and for users who have it. Fortunately, our "otherBSD" friends have drivers that support the most popular of desktop hardware monitoring chips. It seems that the drivers in OpenBSD, NetBSD and DragonflyBSD (unsurprisingly) share most of the code and are collectively maintained. lm(4) driver supports Winbond/Nuvoton lineage, while it(4) supports ITE chips. For me personally these two drivers cover hardware monitoring in 100% of machines I regularly use (Gigabyte and Asus desktops). So I would like to import these drivers into FreeBSD with as minimal code changes as possible. That should make future imports and multi-way code sharing much easier. The problem is these drivers make use of some common infrastructure known as "Sensors Framework". This "Sensors Framework" (along with the drivers) was once ported to FreeBSD during a GSoC project by Constantine A. Murenin. It even was committed to the tree, but then reverted. That happened because the architecture and design of the "Sensors Framework" framework didn't match a concept of a Sensors Framework that FreeBSD developers had in mind, nor their expectations about such a framework. I have been maintaining the code in my own git repository and regularly rebased it on top of recent head. Now I want to re-use the work of others and to re-introduce the code into the tree. But I do _not_ want to call it a "Sensors Framework". Especially I do not want to call it _the_ "Sensors Framework". I do not want to say that the code implements some vision. I do not plan to convert any other drivers to make use of the code. I do not propose any policy that the future drivers should be based on the code. At this moment I do not plan to commit any other drivers besides lm(4) and it(4), but I won't object if someone else does it either. I do not plan to work on any "smart" sensor stuff or on controlling hardware via sensors code (setting thresholds, fan speeds, getting interrupts, etc). I will never object to development of a proper SF. I will try to convert the current code to the proper SF when it happens. For me it is just an easy way to get it(4) and lm(4) in the base and to support them with minimal effort. So I would like to call this code "it(4) and lm(4) drivers plus some shared utility code for kernel drivers for a small number of simple sensors". I would like to emphasize "kernel drivers", "small number" and "simple sensors". I like that the code uses sysctl as an interface to access sensors. sysctl is quite convenient to use in kernel. It is very convenient in userland (for command line and programmatic access). It is totally adequate to query a few very simple values. Besides, this is what is already used all over the place albeit in a non-systematic fashion: coretemp(4), amdtemp(4), acpi_thermal(4), etc. So this is the aspect of the code that I would like to keep. Obviously, I would also like to keep the kernel side interfaces for ease of porting of the individual drivers. If any other aspect of the code is too intrusive, abusive or inefficient to be committed in the current form, I am prepared to re-work or remove it (within the above constraints). I do not plan to start committing anything just yet, so this is an advance notice about my intentions. I'll be here to discuss technical and other sides of these plans. Please let me know what you think. Any suggestions, objections, comments, help, etc are very welcome. Thank you in advance. P.S. Please please let's not discuss what happened back then. Please. P.P.S. Of course I will notify cnst, netchild, syrinx and others about the plans. All the attributions will definitely be made/kept. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 16:17:36 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2AC23106566B; Fri, 5 Oct 2012 16:17:36 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id F09F88FC18; Fri, 5 Oct 2012 16:17:35 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 358DBB944; Fri, 5 Oct 2012 12:17:35 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Fri, 5 Oct 2012 11:41:16 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p20; KDE/4.5.5; amd64; ; ) References: <506C385C.3020400@FreeBSD.org> <86a9w1kq94.fsf@ds4.des.no> <20121005133616.GP35915@deviant.kiev.zoral.com.ua> In-Reply-To: <20121005133616.GP35915@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201210051141.16147.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 05 Oct 2012 12:17:35 -0400 (EDT) Cc: Konstantin Belousov , Dag-Erling Sm??rgrav , Dimitry Andric , Garrett Cooper , Andriy Gapon Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 16:17:36 -0000 On Friday, October 05, 2012 9:36:16 am Konstantin Belousov wrote: > On Fri, Oct 05, 2012 at 03:22:31PM +0200, Dag-Erling Sm??rgrav wrote: > > Konstantin Belousov writes: > > > So what ISA additions do you expect to get advantage of by switching > > > to pentium-mmx from 486 ? As I already said, I am not aware of any. > > > > The TSC, for one. MMX, and the ability to use MMX registers to copy > > data. > > TSC is used regardless of the compiler flags, we use it if CPU claims > that TSC is supported, even in usermode. > > Compiler never generates MMX copies. More, in kernel, the manual > FPU context save/restore is needed around the FPU/MMX register file access. I agree with kib. I don't think building i386 releases with > i486 buys you much of anything. Using MMX in the kernel is of dubious value (have to be very careful to use it, and when tested in the past by bde@ for things like bcopy() and bzero() it wasn't a clear win IIRC). Also, for the boot code, the most important thing is size. The text + data + stack for /boot/loader has to all fit below 640k (and the first 40k is reserved by BTX, so you really only have 600k for that, minus any "low" memory consumed by things like PXE ROMs). That is true even on amd64, and won't be any better on x86 until we fully support EFI for booting. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 16:24:40 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2A899106564A; Fri, 5 Oct 2012 16:24:40 +0000 (UTC) (envelope-from lars@e-new.0x20.net) Received: from mail.0x20.net (mail.0x20.net [217.69.76.211]) by mx1.freebsd.org (Postfix) with ESMTP id 7C0C48FC0A; Fri, 5 Oct 2012 16:24:39 +0000 (UTC) Received: from e-new.0x20.net (mail.0x20.net [IPv6:2001:aa8:fffb:1::3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.0x20.net (Postfix) with ESMTPS id E3C386A6007; Fri, 5 Oct 2012 18:24:37 +0200 (CEST) Received: from e-new.0x20.net (localhost [127.0.0.1]) by e-new.0x20.net (8.14.5/8.14.5) with ESMTP id q95GObW9045924; Fri, 5 Oct 2012 18:24:37 +0200 (CEST) (envelope-from lars@e-new.0x20.net) Received: (from lars@localhost) by e-new.0x20.net (8.14.5/8.14.5/Submit) id q95GObYG045671; Fri, 5 Oct 2012 18:24:37 +0200 (CEST) (envelope-from lars) Date: Fri, 5 Oct 2012 18:24:37 +0200 From: Lars Engels To: Andriy Gapon Message-ID: <20121005162437.GF7416@e-new.0x20.net> References: <506F06FA.4050804@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="kb0TSCuX821Ar6UT" Content-Disposition: inline In-Reply-To: <506F06FA.4050804@FreeBSD.org> X-Editor: VIM - Vi IMproved 7.3 X-Operation-System: FreeBSD 8.3-RELEASE-p4 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-arch@FreeBSD.org Subject: Re: drivers for desktop hardware monitoring chips X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 16:24:40 -0000 --kb0TSCuX821Ar6UT Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Oct 05, 2012 at 07:12:42PM +0300, Andriy Gapon wrote: >=20 > Currently FreeBSD is severely lacking in support for basic hardware monit= oring on > desktop-class systems. Such systems typically do not have IPMI or other = similar > types of agents. Instead they usually have a hardware monitoring functio= n in a > Super I/O chip or less frequently in a dedicated chip. > Popular manufacturers of such chips are Nuvoton (formerly Winbond) and IT= E. >=20 > FreeBSD doesn't provide any support for this hardware neither in kernel n= or in > base system userland tools. There are a number of tools in ports (e.g. m= bmon), > but they are of varying quality and most of them are very out-of-date. S= o they do > not properly support recent (and no so recent) hardware. >=20 > The userland tools are dying out mostly because most of other popular ope= rating > systems have built-in support for such hardware and thus there is no ince= ntive to > support any cross-platform userland utilities. Which is not trivial too,= because > of different policies and interfaces to access system/hardware resources. >=20 > So I think that it is time that we also provide some support for this kin= d of > hardware and for users who have it. > Fortunately, our "otherBSD" friends have drivers that support the most po= pular of > desktop hardware monitoring chips. It seems that the drivers in OpenBSD,= NetBSD > and DragonflyBSD (unsurprisingly) share most of the code and are collecti= vely > maintained. > lm(4) driver supports Winbond/Nuvoton lineage, while it(4) supports ITE c= hips. > For me personally these two drivers cover hardware monitoring in 100% of = machines > I regularly use (Gigabyte and Asus desktops). >=20 > So I would like to import these drivers into FreeBSD with as minimal code= changes > as possible. That should make future imports and multi-way code sharing = much easier. >=20 > The problem is these drivers make use of some common infrastructure known= as > "Sensors Framework". This "Sensors Framework" (along with the drivers) w= as once > ported to FreeBSD during a GSoC project by Constantine A. Murenin. It ev= en was > committed to the tree, but then reverted. That happened because the arch= itecture > and design of the "Sensors Framework" framework didn't match a concept of= a > Sensors Framework that FreeBSD developers had in mind, nor their expectat= ions > about such a framework. >=20 > I have been maintaining the code in my own git repository and regularly r= ebased it > on top of recent head. Now I want to re-use the work of others and to > re-introduce the code into the tree. > But I do _not_ want to call it a "Sensors Framework". Especially I do no= t want to > call it _the_ "Sensors Framework". > I do not want to say that the code implements some vision. > I do not plan to convert any other drivers to make use of the code. > I do not propose any policy that the future drivers should be based on th= e code. > At this moment I do not plan to commit any other drivers besides lm(4) an= d it(4), > but I won't object if someone else does it either. > I do not plan to work on any "smart" sensor stuff or on controlling hardw= are via > sensors code (setting thresholds, fan speeds, getting interrupts, etc). > I will never object to development of a proper SF. I will try to convert= the > current code to the proper SF when it happens. >=20 > For me it is just an easy way to get it(4) and lm(4) in the base and to s= upport > them with minimal effort. So I would like to call this code "it(4) and l= m(4) > drivers plus some shared utility code for kernel drivers for a small numb= er of > simple sensors". > I would like to emphasize "kernel drivers", "small number" and "simple se= nsors". >=20 > I like that the code uses sysctl as an interface to access sensors. sysc= tl is > quite convenient to use in kernel. It is very convenient in userland (fo= r command > line and programmatic access). It is totally adequate to query a few ver= y simple > values. Besides, this is what is already used all over the place albeit = in a > non-systematic fashion: coretemp(4), amdtemp(4), acpi_thermal(4), etc. > So this is the aspect of the code that I would like to keep. >=20 > Obviously, I would also like to keep the kernel side interfaces for ease = of > porting of the individual drivers. >=20 > If any other aspect of the code is too intrusive, abusive or inefficient = to be > committed in the current form, I am prepared to re-work or remove it (wit= hin the > above constraints). >=20 > I do not plan to start committing anything just yet, so this is an advanc= e notice > about my intentions. I'll be here to discuss technical and other sides o= f these > plans. >=20 > Please let me know what you think. > Any suggestions, objections, comments, help, etc are very welcome. > Thank you in advance. >=20 > P.S. Please please let's not discuss what happened back then. Please. > P.P.S. Of course I will notify cnst, netchild, syrinx and others about th= e plans. > All the attributions will definitely be made/kept. Can you give us a link to the code in question? --kb0TSCuX821Ar6UT Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlBvCcUACgkQKc512sD3afiRjwCfWAUuARBkr58u9cp2sbm24jaF Yy8An3ru9rCNIBfR0to3kwbJzZ/IPr4O =FCzV -----END PGP SIGNATURE----- --kb0TSCuX821Ar6UT-- From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 16:27:43 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A05DB106564A; Fri, 5 Oct 2012 16:27:43 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 660348FC12; Fri, 5 Oct 2012 16:27:43 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id rp8so2308926pbb.13 for ; Fri, 05 Oct 2012 09:27:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; bh=ux20aodRK6BFxJXthKsgesB7yF/xkACq/pntfgQGhDQ=; b=HcTBfh1JqhdSSMaGaD5/AIzAiZxe1Ebd1H7zQQnqe6qCxH/wcC0456n2Nh8LyvYv5k 9DD67vvtpe2qsVvHYzM5erjKjjXa9Np30u3CX26SP4SzLDvR5/66mVK8KCMGw0T+vBQD VF6grP8JEWlwg+GGL1K/b8YKTATJb0dYy8Lvpc7itSWbRnhoMeJsW/b/n9LJXaiLHezi SxlmELZopq06JTMtCqqMARMF00c0ZiWXkAnzkQFwvJCiFUGjOkFc+m5DgQ4uqvGFxFtR P+8y4eCrymBMDZi0qmX9B/DBkDo/kLkZsXSwiAS1ULOW27aPyqeBoEPU576zjp6eT5yO Z3jw== MIME-Version: 1.0 Received: by 10.68.242.164 with SMTP id wr4mr32311699pbc.41.1349454462832; Fri, 05 Oct 2012 09:27:42 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.68.223.136 with HTTP; Fri, 5 Oct 2012 09:27:42 -0700 (PDT) In-Reply-To: <20121005155441.GL69724@acme.spoerlein.net> References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <86haq9hq2c.fsf@ds4.des.no> <20121005033244.GL35915@deviant.kiev.zoral.com.ua> <86y5jll7kc.fsf@ds4.des.no> <86txu9l4z9.fsf@ds4.des.no> <20121005155441.GL69724@acme.spoerlein.net> Date: Fri, 5 Oct 2012 09:27:42 -0700 X-Google-Sender-Auth: XlkNwO859FKircH6AcpCRBdkVMM Message-ID: From: Adrian Chadd To: =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= , Garrett Cooper , Konstantin Belousov , Dimitry Andric , Andriy Gapon , freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 16:27:43 -0000 On 5 October 2012 08:54, Ulrich Sp=F6rlein wrote: > Seriously? How about we leave i386 as is for the embedded folks and just > move forward with amd64. Any improvements made on our i386 support are > useless to about 98% of our users. It's 2012 people ... You can pry i386 from my cold, dead hands. Especially if intel tablets and such appear. I still install i386 on most of my laptop/netbook devices because they don't come with more than 2GB of RAM. Embedded, indeed.. Adrian From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 16:41:43 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EBE29106564A for ; Fri, 5 Oct 2012 16:41:43 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 46AAD8FC12 for ; Fri, 5 Oct 2012 16:41:42 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA00953; Fri, 05 Oct 2012 19:41:35 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <506F0DBE.6050307@FreeBSD.org> Date: Fri, 05 Oct 2012 19:41:34 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:15.0) Gecko/20120911 Thunderbird/15.0.1 MIME-Version: 1.0 To: Lars Engels References: <506F06FA.4050804@FreeBSD.org> <20121005162437.GF7416@e-new.0x20.net> In-Reply-To: <20121005162437.GF7416@e-new.0x20.net> X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: freebsd-arch@FreeBSD.org Subject: Re: drivers for desktop hardware monitoring chips X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 16:41:44 -0000 on 05/10/2012 19:24 Lars Engels said the following: > Can you give us a link to the code in question? Yes, of course. Original code can be found here: http://wiki.freebsd.org/GSoC2007/cnst-sensors What I maintain can be found here: https://github.com/avg-I/freebsd/commit/669e06d1da9e18309243630ff86bd106957ffa1b It's that commit and three commits after it. Sorry, it's not a separate branch and I haven't pushed my latest tree to github yet, but I plan to do that soon. Basically it's the same code just adapted to the latest tree. Plus some enhancements for it(4) driver. I also have support for a new lm(4) chip in the works. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 17:58:50 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6E1711065672; Fri, 5 Oct 2012 17:58:50 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 448C78FC08; Fri, 5 Oct 2012 17:58:50 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 33678B939; Fri, 5 Oct 2012 13:58:35 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Fri, 5 Oct 2012 13:58:28 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p20; KDE/4.5.5; amd64; ; ) References: <506F06FA.4050804@FreeBSD.org> In-Reply-To: <506F06FA.4050804@FreeBSD.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201210051358.28286.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 05 Oct 2012 13:58:35 -0400 (EDT) Cc: Andriy Gapon Subject: Re: drivers for desktop hardware monitoring chips X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 17:58:50 -0000 On Friday, October 05, 2012 12:12:42 pm Andriy Gapon wrote: > > Currently FreeBSD is severely lacking in support for basic hardware monitoring on > desktop-class systems. Such systems typically do not have IPMI or other similar > types of agents. Instead they usually have a hardware monitoring function in a > Super I/O chip or less frequently in a dedicated chip. > Popular manufacturers of such chips are Nuvoton (formerly Winbond) and ITE. In general I think this is a good idea. We did head too far off in the weeds last time. It would be nice to eventually build a more unified framework on top of this that also includes things like coretemp and IPMI sensors, but that can be a layer on top of various existing drivers, including on top of this IMO. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 20:14:08 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7D018106566B; Fri, 5 Oct 2012 20:14:08 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 377948FC17; Fri, 5 Oct 2012 20:14:08 +0000 (UTC) Received: from critter.freebsd.dk (critter-phk.freebsd.dk [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 31CB93B692; Fri, 5 Oct 2012 20:06:49 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id q95K6qfg001776; Fri, 5 Oct 2012 20:06:53 GMT (envelope-from phk@phk.freebsd.dk) To: Andriy Gapon From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 05 Oct 2012 19:12:42 +0300." <506F06FA.4050804@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1 Date: Fri, 05 Oct 2012 20:06:52 +0000 Message-ID: <1775.1349467612@critter.freebsd.dk> Cc: freebsd-arch@FreeBSD.org Subject: Re: drivers for desktop hardware monitoring chips X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 20:14:08 -0000 In message <506F06FA.4050804@FreeBSD.org>, Andriy Gapon writes: >Especially I do not want to call it _the_ "Sensors Framework". It doesn't really matter what you call it, it still sucks :-) See also: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=1863154+0+archive/2002/freebsd-current/20021006.freebsd-current -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 20:22:41 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7B7DA106564A; Fri, 5 Oct 2012 20:22:41 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 374E18FC12; Fri, 5 Oct 2012 20:22:40 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id rp8so2570904pbb.13 for ; Fri, 05 Oct 2012 13:22:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=3OLMDUqrwTyaJrRaDTf+pwO+vp76rxgkZev9dWL9UNs=; b=BE6XjzJf99E+TTaVlIObkwgUq7S8ozHyHz+ney1nuh+d4oS0h8lPdkSkr/q7Xmb9PB iSa1pM8oQjEHFJbiNqADggsXoV8MJ2f1tMCuaDmPmBHyDBOsWVkuj2F6AdY6N3PZMITx ASBlOsQDsqjczB0kbRXMyd8Rm1rN8DqvEqab4H56Pkah0ufoeSyhsjz0LWphnwdvt5Nn LFGSMZAMQHTpoh2fQRJL4Qptzue5+bdaAarm/RdV/RphqamCmMbJtirBVy0uWdB2fBB8 IQh0Xr3ws/wi21jB7WWT13RsE9Tpurj+w0C57nQYjvc2qq1OhYrfekmpA0HZdPXpISyc EF0w== Received: by 10.68.213.1 with SMTP id no1mr16677709pbc.123.1349468554098; Fri, 05 Oct 2012 13:22:34 -0700 (PDT) Received: from fuji-wireless.local (c-24-19-191-56.hsd1.wa.comcast.net. [24.19.191.56]) by mx.google.com with ESMTPS id w10sm6499046paz.22.2012.10.05.13.22.29 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 05 Oct 2012 13:22:33 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=iso-8859-1 From: Garrett Cooper In-Reply-To: Date: Fri, 5 Oct 2012 13:23:10 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <86haq9hq2c.fsf@ds4.des.no> <20121005033244.GL35915@deviant.kiev.zoral.com.ua> <86y5jll7kc.fsf@ds4.des.no> <86txu9l4z9.fsf@ds4.des.no> <20121005155441.GL69724@acme.spoerlein.net> To: Adrian Chadd X-Mailer: Apple Mail (2.1278) Cc: Konstantin Belousov , =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= , Dimitry Andric , Andriy Gapon , freebsd-arch@freebsd.org Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 20:22:41 -0000 On Oct 5, 2012, at 9:27 AM, Adrian Chadd wrote: > On 5 October 2012 08:54, Ulrich Sp=F6rlein wrote: >=20 >> Seriously? How about we leave i386 as is for the embedded folks and = just >> move forward with amd64. Any improvements made on our i386 support = are >> useless to about 98% of our users. It's 2012 people ... >=20 > You can pry i386 from my cold, dead hands. Especially if intel tablets > and such appear. >=20 > I still install i386 on most of my laptop/netbook devices because they > don't come with more than 2GB of RAM. >=20 > Embedded, indeed.. +1 for my netbook (2GB RAM) and router/gateway/firewall (4GB = RAM). With Atom processors you're pretty much stuck with i386, in part = because amd64 emulation mode still sucks a lot. Thanks, -Garrett= From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 20:42:45 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EE9D5106564A; Fri, 5 Oct 2012 20:42:44 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) by mx1.freebsd.org (Postfix) with ESMTP id 801DE8FC14; Fri, 5 Oct 2012 20:42:44 +0000 (UTC) Received: from rbpbp.gid.co.uk (80-46-130-69.static.dsl.as9105.com [80.46.130.69]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id q95Kg2Sr017790; Fri, 5 Oct 2012 21:42:02 +0100 (BST) (envelope-from rb@gid.co.uk) Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=iso-8859-1 From: Bob Bishop In-Reply-To: Date: Fri, 5 Oct 2012 21:41:57 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <5E8C160D-5B57-4D31-9052-3AE7FEE64313@gid.co.uk> References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <86haq9hq2c.fsf@ds4.des.no> <20121005033244.GL35915@deviant.kiev.zoral.com.ua> <86y5jll7kc.fsf@ds4.des.no> <86txu9l4z9.fsf@ds4.des.no> <20121005155441.GL69724@acme.spoerlein.net> To: Garrett Cooper X-Mailer: Apple Mail (2.1283) Cc: Adrian Chadd , Andriy Gapon , freebsd-arch@freebsd.org, Konstantin Belousov , =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= , Dimitry Andric Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 20:42:45 -0000 Hi, On 5 Oct 2012, at 21:23, Garrett Cooper wrote: > With Atom processors you're pretty much stuck with i386, in part = because amd64 emulation mode still sucks a lot. Can't agree with that in general, we've had no problems with the D525 = running amd64 (mainly for the benefit of ZFS). It probably depends = exactly which Atom you are looking at - they differ quite a bit. -- Bob Bishop rb@gid.co.uk From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 21:01:58 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2E7D8106566B; Fri, 5 Oct 2012 21:01:58 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) by mx1.freebsd.org (Postfix) with ESMTP id DD4F38FC08; Fri, 5 Oct 2012 21:01:57 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id bi1so2533941pad.13 for ; Fri, 05 Oct 2012 14:01:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=PPmgV5gclEXkewMAMmgKxVlf8pApAEfEaJFXSBb40h0=; b=vHxkqpZ5ZjRyOZaWd4qbPur7d8HFlB7e75+Yt1TpdWnrmqGvY0x7UOd2jLUHXWAF0K O+fYfcmafzhUHbKeO2S4JRCFITHKaNJ0p/coqhE5nDN0iOtaR9Oan0lEGCbUnNpM7beK quljaC8Gm5agWC2uOOtlsIUmKAJWWlnAqdu4fa8Fk0zWGelyI342UE2JEU91RWhVBkNh qZEHm9HZvdqwQ3BVoWCgTFDl5GVflLm6S9MHfTJtKNCT0wMfBOlDFE+tiuCcAXMobB+K 1IJ4v/7bCSIgfvg58V8JAEiPPnzNULAZq7AWfxZQJLOHM0T1jPeNn4CFNyytxW8W9697 PrWA== Received: by 10.66.88.198 with SMTP id bi6mr24636283pab.23.1349470917448; Fri, 05 Oct 2012 14:01:57 -0700 (PDT) Received: from fuji-wireless.local (c-24-19-191-56.hsd1.wa.comcast.net. [24.19.191.56]) by mx.google.com with ESMTPS id c5sm6548212pay.5.2012.10.05.14.01.52 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 05 Oct 2012 14:01:55 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=iso-8859-1 From: Garrett Cooper In-Reply-To: <5E8C160D-5B57-4D31-9052-3AE7FEE64313@gid.co.uk> Date: Fri, 5 Oct 2012 14:02:33 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <3E03A2A1-A46A-4601-A69F-AB45329F7361@gmail.com> References: <506C385C.3020400@FreeBSD.org> <506DEB4C.5020508@andric.com> <86haq9hq2c.fsf@ds4.des.no> <20121005033244.GL35915@deviant.kiev.zoral.com.ua> <86y5jll7kc.fsf@ds4.des.no> <86txu9l4z9.fsf@ds4.des.no> <20121005155441.GL69724@acme.spoerlein.net> <5E8C160D-5B57-4D31-9052-3AE7FEE64313@gid.co.uk> To: Bob Bishop X-Mailer: Apple Mail (2.1278) Cc: Adrian Chadd , Andriy Gapon , freebsd-arch@freebsd.org, Konstantin Belousov , =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= , Dimitry Andric Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 21:01:58 -0000 On Oct 5, 2012, at 1:41 PM, Bob Bishop wrote: > Hi, >=20 > On 5 Oct 2012, at 21:23, Garrett Cooper wrote: >=20 >> With Atom processors you're pretty much stuck with i386, in part = because amd64 emulation mode still sucks a lot. >=20 > Can't agree with that in general, we've had no problems with the D525 = running amd64 (mainly for the benefit of ZFS). It probably depends = exactly which Atom you are looking at - they differ quite a bit. Ok. The generation I'm running at least (D4xx) seemed to have = issues with amd64 emulation (slow). Also, some/many PCHs/boards don't = support over 4GB with Atom, which is a bit discouraging if you're trying = to go 64-bit. Thanks, -Garrett= From owner-freebsd-arch@FreeBSD.ORG Fri Oct 5 22:44:42 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A5B84106567B; Fri, 5 Oct 2012 22:44:42 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail02.syd.optusnet.com.au (mail02.syd.optusnet.com.au [211.29.132.183]) by mx1.freebsd.org (Postfix) with ESMTP id 26B3E8FC0C; Fri, 5 Oct 2012 22:44:41 +0000 (UTC) Received: from c122-106-157-84.carlnfd1.nsw.optusnet.com.au (c122-106-157-84.carlnfd1.nsw.optusnet.com.au [122.106.157.84]) by mail02.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id q95MiHTn032665 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 6 Oct 2012 08:44:18 +1000 Date: Sat, 6 Oct 2012 08:44:17 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: John Baldwin In-Reply-To: <201210051141.16147.jhb@freebsd.org> Message-ID: <20121006072636.V978@besplex.bde.org> References: <506C385C.3020400@FreeBSD.org> <86a9w1kq94.fsf@ds4.des.no> <20121005133616.GP35915@deviant.kiev.zoral.com.ua> <201210051141.16147.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Garrett Cooper , Andriy Gapon , freebsd-arch@freebsd.org, Konstantin Belousov , Dag-Erling Sm??rgrav , Dimitry Andric Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2012 22:44:42 -0000 On Fri, 5 Oct 2012, John Baldwin wrote: > On Friday, October 05, 2012 9:36:16 am Konstantin Belousov wrote: >> On Fri, Oct 05, 2012 at 03:22:31PM +0200, Dag-Erling Sm??rgrav wrote: >>> Konstantin Belousov writes: >>>> So what ISA additions do you expect to get advantage of by switching >>>> to pentium-mmx from 486 ? As I already said, I am not aware of any. >>> >>> The TSC, for one. MMX, and the ability to use MMX registers to copy >>> data. >> >> TSC is used regardless of the compiler flags, we use it if CPU claims >> that TSC is supported, even in usermode. >> >> Compiler never generates MMX copies. More, in kernel, the manual >> FPU context save/restore is needed around the FPU/MMX register file access. 1. The TSC provides no significant performance advantage for boot code. In the kernel it is very difficult to use, and provides few advantages for pentium-mmx. It takes about core2 or later for it to be P-state invariant. Then it is not quite so difficult to use, and provides some advantages. 2. MMX for copying data provides no signifant performance advantage for boot code. In the kernel, it is difficult to use, and provides few advantages for pentium-mmx. MMX registers are only 64 bits wide, and the copying speed tends to be limited more by (lack of) caches and write buffers than by the registers used. SSE registers provide larger advantages by being 128 bits wide. It takes about an AthlonXP or later for SSE plus enough extensions (at least one movnt* instruction is needed and I think basic SSE doesn't have any). The best method is very machine- and context-dependent. Someone named des removed my hooks for plugging in the best known copying routines at runtime. I was happy to see them gone, since they are too compicated to used. There would have to be about 100 different versions for each of bcopy, bzero, copyin and copyout (memcpy and friends are intentionally not optimized, since use use of them for large data asks for slowness). I only tested about 40 different versions of bcopy and 20 of bzero. > I agree with kib. I don't think building i386 releases with > i486 buys > you much of anything. Using MMX in the kernel is of dubious value (have to > be very careful to use it, and when tested in the past by bde@ for things like > bcopy() and bzero() it wasn't a clear win IIRC). Here are results of a current run of old test code: on core2 (ref10-i386): results only for a data size of 4K (for much smaller sizes, simple methods are best, and for much larger sizes, all reasonable methods are limited by the speed of main memory and cache overheads, and all reasonable methods have the same speed, except ones using movnt* are faster since they bypass the caches): % copy0: 12146747898 B/s ( 263445 us) (511794241 tsc) (movsl) movsl is a good general method, and on this CPU it is almost twice as fast as all other methods that don't use SSE. (On Athlon64, some of the other non-SSE methods are competitive). % copy1: 7120415120 B/s ( 449412 us) (838775735 tsc) (unroll *4) % copy2: 5773557468 B/s ( 554251 us) (1032266095 tsc) (unroll *4 prefetch) % copy3: 4452898768 B/s ( 718633 us) (1338746402 tsc) (unroll *16 i586-opt) % copy4: 6465613041 B/s ( 494926 us) (921710503 tsc) (unroll *16 i586-opt prefetch) % copy5: 6328337902 B/s ( 505662 us) (942113053 tsc) (unroll *16 i586-opx prefetch) % copy6: 4838090285 B/s ( 661418 us) (1231845839 tsc) (unroll *8 prefetch 4) % copy7: 7290755322 B/s ( 438912 us) (817908588 tsc) (unroll 64 fp++) % copy8: 6463210196 B/s ( 495110 us) (922004965 tsc) (unroll 128 fp i-prefetch) % copy9: 7264439208 B/s ( 440502 us) (820443267 tsc) (unroll 64 fp reordered) % copyA: 7298770613 B/s ( 438430 us) (816486286 tsc) (unroll 256 fp reordered++) % copyB: 7296257704 B/s ( 438581 us) (816792606 tsc) (unroll 512 fp reordered++) % copyC: 700413769 B/s (4568728 us) (8509304678 tsc) (Terje cksum) % copyD: 6266866684 B/s ( 510622 us) (951099730 tsc) (kernel bcopy (unroll 64 fp i-prefetch)) % copyE: 6479962740 B/s ( 493830 us) (919570911 tsc) (unroll 64 fp i-prefetch++) Raw (i586-optimized) kernel bcopy (copy9) is 12.5% faster than the non-raw version (copyD) mainly because it is sloppy and doesn't do FPU state switching. % copyF: 6463432123 B/s ( 495093 us) (922068252 tsc) (new kernel bcopy (unroll 64 fp i-prefetch++)) "new" kernel bcopy has some improvements for Pentium1 related to fxch. These make little difference on core2, since the overhead of fxch is pipelined almost out of existence on core2. % copyG: 11128460690 B/s ( 287551 us) (535363248 tsc) (memcpy (movsl)) % copyH: 2494210703 B/s (1282971 us) (2389591890 tsc) (movntps) % copyI: 2283259781 B/s (1401505 us) (2611152194 tsc) (movntps with prefetchnta) % copyJ: 2246123156 B/s (1424677 us) (2662460521 tsc) (movntps with block prefetch) % copyK: 13432566418 B/s ( 238227 us) (443974286 tsc) (movq) % copyL: 11812171705 B/s ( 270907 us) (504438067 tsc) (movq with prefetchnta) % copyM: 12430515361 B/s ( 257431 us) (479327961 tsc) (movq with block prefetch) movq (64 bits through MMX registers) gives the same speed as movsl. But state switching for MMX would probably cost 12.5% like it does for i586- optimized kernel bcopy. % copyQ: 26618974338 B/s ( 120215 us) (223928117 tsc) (movdqa) % copyR: 21855833459 B/s ( 146414 us) (272801830 tsc) (movdqa with prefetchnta) % copyS: 22343716179 B/s ( 143217 us) (266771960 tsc) (movdqa with block prefetch) movdqa (128 bits through SSE registers using an SSE2 instruction) is the only method tested that is significantly faster than movsl (about twice as fast). Here all data is in the L1 cache except possibly for the first iteration (there are several hundred thousand iterations). % copyT: 6627728760 B/s ( 482820 us) (899276378 tsc) (unroll *8 a64-opt) % copyU: 6441859201 B/s ( 496751 us) (925378496 tsc) (unroll *8 a64-opt with prefetchnta) % copyV: 6514737558 B/s ( 491194 us) (914475275 tsc) (unroll *8 a64-opt with block prefetch) % copyW: 2769764215 B/s (1155333 us) (2151805649 tsc) (movnti) % copyX: 2519306152 B/s (1270191 us) (2365494292 tsc) (movnti with prefetchnta) % copyY: 2494284581 B/s (1282933 us) (2389247728 tsc) (movnti with block prefetch) movnti gives the speed of main memory, which is very slow for ref10-i386 (2.7 GB/S). The source is cached, so the only limit should be for writing to the target; movnti prevents this being cached. If the data size were larger than all caches, then movnti would be best and we would hope for a speed of 2.7/2 GB/S; without movnti, we would only hope for 2.7/3 GB/S. It is very difficult for copy routines or callers to know whether movnti should be used to possibly get this speedup by a factor of 1.5 for large data at the possible cost of a speed-down by a factor of 10 for small data. Some systems have relatively faster main memory and it is clear that movnti is less good for them. % copyZ: 18939281846 B/s ( 168961 us) (314562465 tsc) (i686_memcpy( movdqa)) % copya: 19157661568 B/s ( 167035 us) (311110842 tsc) (~i686_memcpy (movaps)) > Also, for the boot code, the most important thing is size. The text + data + > stack for /boot/loader has to all fit below 640k (and the first 40k is > reserved by BTX, so you really only have 600k for that, minus any "low" memory > consumed by things like PXE ROMs). That is true even on amd64, and won't be > any better on x86 until we fully support EFI for booting. Compiling boot code for newer processors would mainly break it for emergency use on older processors. Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Oct 6 07:28:39 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8BAD31065672 for ; Sat, 6 Oct 2012 07:28:39 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id F11C78FC0A for ; Sat, 6 Oct 2012 07:28:38 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id e12so1563547lag.13 for ; Sat, 06 Oct 2012 00:28:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=DiDkxlHakPSASztCwJDkbZohJ8H6hym2agAy2ugUBHw=; b=YONvmtDo+AeLhIppxF6qvRNlmO+jVuJ8tFGcxOYcUC9KZcA84ptsHuO66c7CLZIdFj Sr0opFlELJWi+EJPLEqKPLv3k9lITGmB/S/X8Rhym0I0WcNSDuxSSVYSEeZdIZqfQ/Dm cIV98o+V7ww+StO8WNHENCU5ULwrwRdMNSCCc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=DiDkxlHakPSASztCwJDkbZohJ8H6hym2agAy2ugUBHw=; b=Pgd67sBZcQQWr1TOCEuk71aoF1i8EqQrNB1LeldjfnfViUNH9ElzUpPr7vKEfRj/ga gXQlOVFhzZ6ShP2c7UrMARl3YR0KRCLic4hpu5Ftp+VqxRCTX+1gmWiGhS9Se/ikckYy SJbXVZrb/+3rQwyyKtpL0avEB/n91XP1/HeU8E9pgIOLURqbuhh+DnqvzBTxPKk+j8Jo ojRspAP8tkdNM0NKqtG+0Ggr7KB2czKENWjJ+KBJjYLIg6AvtdzXaK/9ihC7NHJNaU2y 0jwUOsjHC+Hg+V0Q0/DD1fAzmPFjnctOOdH5JblI59g4B2hYggeYj0jaBREzYH60ba3e tX8g== MIME-Version: 1.0 Received: by 10.152.47.112 with SMTP id c16mr8694130lan.50.1349508517759; Sat, 06 Oct 2012 00:28:37 -0700 (PDT) Received: by 10.112.87.106 with HTTP; Sat, 6 Oct 2012 00:28:37 -0700 (PDT) In-Reply-To: <20121006072636.V978@besplex.bde.org> References: <506C385C.3020400@FreeBSD.org> <86a9w1kq94.fsf@ds4.des.no> <20121005133616.GP35915@deviant.kiev.zoral.com.ua> <201210051141.16147.jhb@freebsd.org> <20121006072636.V978@besplex.bde.org> Date: Sat, 6 Oct 2012 00:28:37 -0700 Message-ID: From: Peter Wemm To: Bruce Evans Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQmRqSnPIp4vtAMebq7sL1Rm6FbkaDBsqQrrqPtC0UWlVkuOBeOLO13K64w5U4nqBZ1hNHEt Cc: Garrett Cooper , Andriy Gapon , freebsd-arch@freebsd.org, Konstantin Belousov , Dag-Erling Sm??rgrav , Dimitry Andric Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Oct 2012 07:28:39 -0000 On Fri, Oct 5, 2012 at 3:44 PM, Bruce Evans wrote: [..] > Here are results of a current run of old test code: on core2 > (ref10-i386): results only for a data size of 4K (for much smaller > sizes, simple methods are best, and for much larger sizes, all > reasonable methods are limited by the speed of main memory and cache > overheads, and all reasonable methods have the same speed, except ones > using movnt* are faster since they bypass the caches): A while ago I experimented with switching 32 bit binaries into 64 bit mode while running under a 64 bit OS for things like data copies. The differences between 32 and 64 bit used to be substantial for the dumber data copy methods. And of course the overheads of getting into and out of 64 bit mode at the time was prohibitive on an Intel processor (compared to an AMD). Short version to explain the concept: bcopy64: pushl %ebx pushl %esi pushl %edi call base base: popl %esi movl %esi,%edx addl $to64-base,%edx pushl $43 /* $GSEL(GUCODE_SEL, SEL_UPL) */ pushl %edx lretl .code64 to64: movq %rsi,%r9 addq $to32-base,%r9 movq 16(%rsp),%rsi /* src */ movq 24(%rsp),%rdi /* dst */ movq 32(%rsp),%rdx /* len */ [... 64 bit bcopy trimmed...] 2: /* Jump back to 32 bit code segment */ pushq $27 /* GSEL(GUCODE32_UPL, SEL_UPL) */ pushq %r9 lretq .code32 .p2align 4 to32: popl %edi popl %esi popl %ebx ret Of course, this requires regular i386 code running on an amd64 kernel. At the time it was quite safe because signal delivery would reset %cs to deliver signals in 32 bit mode and all 64 bits of all registers were context switched, even for a 32 bit application. This was part of a larger skunkworks project I did at work called "EMM64". (A reference to the old dos EMM386). It was a set of >4GB extensions and management code for a regular 32 bit app. One of the things we used it for was to mmap a 16GB file above the 4GB mark and use some hand-rolled hash search code. This allowed us to use some very large hashed key/value stores (before the days of things like memcached). Highlights: mmap64() etc from a 32 bit process.. to put data above 4GB. call64(): quick and easy trampoline to minimize assembler code. dlopen64(), dlsym64(), dlcall64() etc: basically allowed you to compile severely limited 64 bit .so file and (relatively) easily call it from an otherwise unmodified 32 bit application. The use case was to sneak some memory/performace critical patches into certain 32 bit apps that couldn't/wouldn't be recompiled at work. Allocating space above the 4GB mark was entirely different than simply running a small chunk of 64 bit code in 4GB of address space. I had a kernel module that "patched" a few things in the VM and elf32 wrappers. If it crashed.. well.. elf32 core files didn't express 64 bit mappings too well. Yeah, it was quite a hack.. by all definitions of the term. In any case, I'd be curious to know if people could hand tune a hybrid set of 32+64 data manipulation code to outperform pure i386. It was clear at the time that badly written hybrid code outperformed badly written 32 bit code. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell From owner-freebsd-arch@FreeBSD.ORG Sat Oct 6 08:49:45 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 93ECD106566B for ; Sat, 6 Oct 2012 08:49:45 +0000 (UTC) (envelope-from stefan@fafoe.narf.at) Received: from fep17.mx.upcmail.net (fep17.mx.upcmail.net [62.179.121.37]) by mx1.freebsd.org (Postfix) with ESMTP id D60FC8FC08 for ; Sat, 6 Oct 2012 08:49:44 +0000 (UTC) Received: from edge03.upcmail.net ([192.168.13.238]) by viefep17-int.chello.at (InterMail vM.8.01.05.05 201-2260-151-110-20120111) with ESMTP id <20121006084937.SLZF5175.viefep17-int.chello.at@edge03.upcmail.net> for ; Sat, 6 Oct 2012 10:49:37 +0200 Received: from mole.fafoe.narf.at ([80.109.55.137]) by edge03.upcmail.net with edge id 7kpd1k00H2xdvHc03kpduB; Sat, 06 Oct 2012 10:49:37 +0200 X-SourceIP: 80.109.55.137 Received: by mole.fafoe.narf.at (Postfix, from userid 1001) id D63E56D430; Sat, 6 Oct 2012 10:49:36 +0200 (CEST) Date: Sat, 6 Oct 2012 10:49:36 +0200 From: Stefan Farfeleder To: arch@freebsd.org Message-ID: <20121006084936.GA1434@mole.fafoe.narf.at> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="bp/iNruPH9dso1Pn" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Subject: Allowing \xxx in fstab X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Oct 2012 08:49:45 -0000 --bp/iNruPH9dso1Pn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, I want to add support to specify device names containing spaces in fstab. Filesystem labels often contain spaces and we create corresponding devices names, e.g., /dev/msdosfs/FOO BAR. Currently there's no way to mount such devices via fstab. The standard way of doing that in Linux seems to be to encode the space with '\040' in fstab, so I followed that approach. There's a patch in PR 117687 (http://www.freebsd.org/cgi/query-pr.cgi?pr=117687) that uses strunvis() but I feel not very confident with the function as it translates more things than just \xxx. Comments? Stefan --bp/iNruPH9dso1Pn Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="fstab-octal-escape-sequence.diff" Index: lib/libc/gen/fstab.c =================================================================== --- lib/libc/gen/fstab.c (revision 241046) +++ lib/libc/gen/fstab.c (working copy) @@ -107,6 +107,25 @@ _fs_fstab.fs_spec = buf; } +static void +unescapeoctal(char *s) +{ + char *p; + +#define oct(x) ((x) >= '0' && (x) <= '7') + /* Replace \xxx with the corresponding character. */ + p = s; + while (*s != '\0') + if (s[0] == '\\' && oct(s[1]) && oct(s[2]) && oct(s[3])) { + *p++ = (s[1] - '0') * 64 + (s[2] - '0') * 8 + + (s[3] - '0'); + s += 4; + } else + *p++ = *s++; + *p = '\0'; +#undef oct +} + static int fstabscan() { @@ -150,6 +169,7 @@ while ((cp = strsep(&p, " \t\n")) != NULL && *cp == '\0') ; _fs_fstab.fs_spec = cp; + unescapeoctal(_fs_fstab.fs_spec); if (!_fs_fstab.fs_spec || *_fs_fstab.fs_spec == '#') continue; while ((cp = strsep(&p, " \t\n")) != NULL && *cp == '\0') --bp/iNruPH9dso1Pn-- From owner-freebsd-arch@FreeBSD.ORG Sat Oct 6 08:57:54 2012 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 412A9106566B; Sat, 6 Oct 2012 08:57:54 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id EE89A8FC16; Sat, 6 Oct 2012 08:57:53 +0000 (UTC) Received: from critter.freebsd.dk (critter-phk.freebsd.dk [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id C77653B612; Sat, 6 Oct 2012 08:57:52 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id q968vwNM002133; Sat, 6 Oct 2012 08:57:58 GMT (envelope-from phk@phk.freebsd.dk) To: Stefan Farfeleder From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sat, 06 Oct 2012 10:49:36 +0200." <20121006084936.GA1434@mole.fafoe.narf.at> Content-Type: text/plain; charset=ISO-8859-1 Date: Sat, 06 Oct 2012 08:57:58 +0000 Message-ID: <2132.1349513878@critter.freebsd.dk> Cc: arch@FreeBSD.org Subject: Re: Allowing \xxx in fstab X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Oct 2012 08:57:54 -0000 In message <20121006084936.GA1434@mole.fafoe.narf.at>, Stefan Farfeleder writes : >+static void >+unescapeoctal(char *s) Use unvis(3) ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sat Oct 6 09:12:04 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0280C1065670 for ; Sat, 6 Oct 2012 09:12:04 +0000 (UTC) (envelope-from stefan@fafoe.narf.at) Received: from fep13.mx.upcmail.net (fep13.mx.upcmail.net [62.179.121.33]) by mx1.freebsd.org (Postfix) with ESMTP id 440E68FC08 for ; Sat, 6 Oct 2012 09:12:03 +0000 (UTC) Received: from edge01.upcmail.net ([192.168.13.236]) by viefep13-int.chello.at (InterMail vM.8.01.05.05 201-2260-151-110-20120111) with ESMTP id <20121006091155.HPKZ3518.viefep13-int.chello.at@edge01.upcmail.net>; Sat, 6 Oct 2012 11:11:55 +0200 Received: from mole.fafoe.narf.at ([80.109.55.137]) by edge01.upcmail.net with edge id 7lBv1k00b2xdvHc01lBvUg; Sat, 06 Oct 2012 11:11:55 +0200 X-SourceIP: 80.109.55.137 Received: by mole.fafoe.narf.at (Postfix, from userid 1001) id 361E16D430; Sat, 6 Oct 2012 11:11:55 +0200 (CEST) Date: Sat, 6 Oct 2012 11:11:55 +0200 From: Stefan Farfeleder To: Poul-Henning Kamp Message-ID: <20121006091154.GB1434@mole.fafoe.narf.at> References: <20121006084936.GA1434@mole.fafoe.narf.at> <2132.1349513878@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2132.1349513878@critter.freebsd.dk> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@FreeBSD.org Subject: Re: Allowing \xxx in fstab X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Oct 2012 09:12:04 -0000 On Sat, Oct 06, 2012 at 08:57:58AM +0000, Poul-Henning Kamp wrote: > In message <20121006084936.GA1434@mole.fafoe.narf.at>, Stefan Farfeleder writes > : > > >+static void > >+unescapeoctal(char *s) > > > Use unvis(3) ? The unvis functions unfortunately cannot be told be decode only octal escapes. But if everyone agrees that strings like "\^C" or "\$" will never occur in device names, I'm happy to use them. Stefan From owner-freebsd-arch@FreeBSD.ORG Sat Oct 6 12:23:23 2012 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E0C661065672; Sat, 6 Oct 2012 12:23:23 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 6D56E8FC08; Sat, 6 Oct 2012 12:23:23 +0000 (UTC) Received: from critter.freebsd.dk (critter-phk.freebsd.dk [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 7236B3B754; Sat, 6 Oct 2012 12:23:21 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id q96CNQwe003021; Sat, 6 Oct 2012 12:23:26 GMT (envelope-from phk@phk.freebsd.dk) To: Stefan Farfeleder From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sat, 06 Oct 2012 11:11:55 +0200." <20121006091154.GB1434@mole.fafoe.narf.at> Content-Type: text/plain; charset=ISO-8859-1 Date: Sat, 06 Oct 2012 12:23:26 +0000 Message-ID: <3020.1349526206@critter.freebsd.dk> Cc: arch@FreeBSD.org Subject: Re: Allowing \xxx in fstab X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Oct 2012 12:23:24 -0000 In message <20121006091154.GB1434@mole.fafoe.narf.at>, Stefan Farfeleder writes : >On Sat, Oct 06, 2012 at 08:57:58AM +0000, Poul-Henning Kamp wrote: >> In message <20121006084936.GA1434@mole.fafoe.narf.at>, Stefan Farfeleder writes >> : >> >> >+static void >> >+unescapeoctal(char *s) >> >> >> Use unvis(3) ? > >The unvis functions unfortunately cannot be told be decode only octal >escapes. But if everyone agrees that strings like "\^C" or "\$" will >never occur in device names, I'm happy to use them. Or improve unvis ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sat Oct 6 17:39:42 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5E1EF106564A; Sat, 6 Oct 2012 17:39:42 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id E81888FC0C; Sat, 6 Oct 2012 17:39:41 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q96HdeAA037645; Sat, 6 Oct 2012 20:39:40 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q96HdSZD097497; Sat, 6 Oct 2012 20:39:28 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q96HdQ79097496; Sat, 6 Oct 2012 20:39:26 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 6 Oct 2012 20:39:26 +0300 From: Konstantin Belousov To: Peter Wemm Message-ID: <20121006173926.GT35915@deviant.kiev.zoral.com.ua> References: <506C385C.3020400@FreeBSD.org> <86a9w1kq94.fsf@ds4.des.no> <20121005133616.GP35915@deviant.kiev.zoral.com.ua> <201210051141.16147.jhb@freebsd.org> <20121006072636.V978@besplex.bde.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ubEh+iSFTuEFLNSo" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Garrett Cooper , Andriy Gapon , freebsd-arch@freebsd.org, Dag-Erling Sm??rgrav , Dimitry Andric Subject: Re: x86 boot code build X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Oct 2012 17:39:42 -0000 --ubEh+iSFTuEFLNSo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Oct 06, 2012 at 12:28:37AM -0700, Peter Wemm wrote: > On Fri, Oct 5, 2012 at 3:44 PM, Bruce Evans wrote: > [..] > > Here are results of a current run of old test code: on core2 > > (ref10-i386): results only for a data size of 4K (for much smaller > > sizes, simple methods are best, and for much larger sizes, all > > reasonable methods are limited by the speed of main memory and cache > > overheads, and all reasonable methods have the same speed, except ones > > using movnt* are faster since they bypass the caches): >=20 > A while ago I experimented with switching 32 bit binaries into 64 bit > mode while running under a 64 bit OS for things like data copies. The > differences between 32 and 64 bit used to be substantial for the > dumber data copy methods. And of course the overheads of getting into > and out of 64 bit mode at the time was prohibitive on an Intel > processor (compared to an AMD). >=20 > Short version to explain the concept: >=20 > bcopy64: > pushl %ebx > pushl %esi > pushl %edi > call base > base: > popl %esi > movl %esi,%edx > addl $to64-base,%edx > pushl $43 /* $GSEL(GUCODE_SEL, SEL_UPL) */ > pushl %edx > lretl > .code64 > to64: > movq %rsi,%r9 > addq $to32-base,%r9 > movq 16(%rsp),%rsi /* src */ > movq 24(%rsp),%rdi /* dst */ > movq 32(%rsp),%rdx /* len */ > [... 64 bit bcopy trimmed...] >=20 > 2: > /* Jump back to 32 bit code segment */ > pushq $27 /* GSEL(GUCODE32_UPL, SEL_UPL) */ > pushq %r9 > lretq > .code32 > .p2align 4 > to32: > popl %edi > popl %esi > popl %ebx > ret >=20 > Of course, this requires regular i386 code running on an amd64 kernel. > At the time it was quite safe because signal delivery would reset %cs > to deliver signals in 32 bit mode and all 64 bits of all registers > were context switched, even for a 32 bit application. This is still true. We always set %cs, %ss, %ds and %es for the signal handlers, as well as save the full 64bit register file. The recent linuxish x32 ABI might be relevant for what you tried to do. If only they did not mandated bastardized kernel ABI. --ubEh+iSFTuEFLNSo Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlBwbM4ACgkQC3+MBN1Mb4hpKgCgre87/SRosswkltWnRFKVU2/z 3bEAoNB+j9b8M9B3E4yS2QZfcPIFIeIU =8sW5 -----END PGP SIGNATURE----- --ubEh+iSFTuEFLNSo-- From owner-freebsd-arch@FreeBSD.ORG Sat Oct 6 19:39:48 2012 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 708F010656D1; Sat, 6 Oct 2012 19:39:48 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 449A98FC20; Sat, 6 Oct 2012 19:39:48 +0000 (UTC) Received: from John-Baldwins-MacBook-Air.local (c-68-39-198-164.hsd1.de.comcast.net [68.39.198.164]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 910B3B91E; Sat, 6 Oct 2012 15:39:47 -0400 (EDT) Message-ID: <50708903.6060209@FreeBSD.org> Date: Sat, 06 Oct 2012 15:39:47 -0400 From: John Baldwin User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20120824 Thunderbird/15.0 MIME-Version: 1.0 To: Stefan Farfeleder References: <20121006084936.GA1434@mole.fafoe.narf.at> <2132.1349513878@critter.freebsd.dk> <20121006091154.GB1434@mole.fafoe.narf.at> In-Reply-To: <20121006091154.GB1434@mole.fafoe.narf.at> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Sat, 06 Oct 2012 15:39:47 -0400 (EDT) Cc: arch@FreeBSD.org, Poul-Henning Kamp Subject: Re: Allowing \xxx in fstab X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Oct 2012 19:39:48 -0000 On 10/6/12 5:11 AM, Stefan Farfeleder wrote: > On Sat, Oct 06, 2012 at 08:57:58AM +0000, Poul-Henning Kamp wrote: >> In message <20121006084936.GA1434@mole.fafoe.narf.at>, Stefan Farfeleder writes >> : >> >>> +static void >>> +unescapeoctal(char *s) >> >> >> Use unvis(3) ? > > The unvis functions unfortunately cannot be told be decode only octal > escapes. But if everyone agrees that strings like "\^C" or "\$" will > never occur in device names, I'm happy to use them. I think you are probably fine to just use strunvis(). It would also be nice to allow simple escaping of spaces (\ ). -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Sat Oct 6 23:48:55 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 802131065670; Sat, 6 Oct 2012 23:48:55 +0000 (UTC) (envelope-from Devin.Teske@fisglobal.com) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) by mx1.freebsd.org (Postfix) with ESMTP id 45A5A8FC0A; Sat, 6 Oct 2012 23:48:53 +0000 (UTC) Received: from smtp.fisglobal.com ([10.132.206.16]) by ltcfislmsgpa01.fnfis.com (8.14.4/8.14.4) with ESMTP id q96Nmr3M021320 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Sat, 6 Oct 2012 18:48:53 -0500 Received: from [10.0.0.103] (10.14.152.61) by smtp.fisglobal.com (10.132.206.16) with Microsoft SMTP Server (TLS) id 14.2.309.2; Sat, 6 Oct 2012 18:48:52 -0500 From: Devin Teske Date: Sat, 6 Oct 2012 16:48:50 -0700 Message-ID: <0655B56F-AD43-402B-872C-568378E650F9@fisglobal.com> To: MIME-Version: 1.0 (Apple Message framework v1283) X-Mailer: Apple Mail (2.1283) X-Originating-IP: [10.14.152.61] X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.7.7855, 1.0.431, 0.0.0000 definitions=2012-10-06_07:2012-10-06, 2012-10-06, 1970-01-01 signatures=0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Devin Teske Subject: New Boot Loader Menu X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Devin Teske List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Oct 2012 23:48:55 -0000 Hello, I've been working on a new boot loader menu system. This is what is in HEAD, CURRENT, and RELENG_9 at the moment: http://twitpic.com/b1pkll/full in color: http://twitpic.com/b1pkz1/full I'd like to propose the following replacement to the above: http://twitpic.com/b1pll5/full in color: http://twitpic.com/b1plxi/full The boot options have been whisked away into a sub-menu (see below): http://twitpic.com/b1pm51/full in color: http://twitpic.com/b1pme8/full What does everybody think? NOTE: This change is not trivial. It took me nearly a month of hacking to p= roduce this _and_ almost 1,000 changed lines of code are required. Features= such as submenus, dynamic menus and menu items, and more were added and I'= m at a point where I can commit this back to the tree. I'm sure we want the= se features, but we have a choice of keeping the menu as-is without any cha= nges _or_ we can choose to use the new features (as exhibited in this propo= sal -- where the boot options are sidled-off into a submenu). --=20 Devin P.S. The hope is to one day utilize _all_ of the features I'm adding to one= day have something like this (a functioning mock-up, but unfinished curren= tly): http://twitter.com/devinteske/status/254419042104909824 _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you.