From owner-freebsd-arch@FreeBSD.ORG Mon Jul 19 21:30:59 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C26D0106564A; Mon, 19 Jul 2010 21:30:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 5BAC58FC23; Mon, 19 Jul 2010 21:30:58 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o6JLUsxe079065 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 20 Jul 2010 00:30:54 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o6JLUs2Y072085; Tue, 20 Jul 2010 00:30:54 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o6JLUs7h072084; Tue, 20 Jul 2010 00:30:54 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 20 Jul 2010 00:30:54 +0300 From: Kostik Belousov To: arch@freebsd.org Message-ID: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Cjvo8w+V8lMs5Jsv" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_40, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: amd64@freebsd.org Subject: uname -m/-p for compat32 binaries X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jul 2010 21:30:59 -0000 --Cjvo8w+V8lMs5Jsv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, I intend to commit the following change, that makes sysctls hw.machine_arch and hw.machine to return "i386" for 32 bit binaries run on amd64. In particular, 32 bit uname -m and uname -p print "i386", that is good for i386 jails on amd64 kernels. I find the change very useful for me, but I wonder why such trivial modification is not yet done. Can anybody note a possible fallout from it ? diff --git a/sys/amd64/amd64/identcpu.c b/sys/amd64/amd64/identcpu.c index 52e7568..68de22b 100644 --- a/sys/amd64/amd64/identcpu.c +++ b/sys/amd64/amd64/identcpu.c @@ -76,8 +76,26 @@ static void print_via_padlock_info(void); =20 int cpu_class; char machine[] =3D "amd64"; -SYSCTL_STRING(_hw, HW_MACHINE, machine, CTLFLAG_RD,=20 - machine, 0, "Machine class"); + +static int +sysctl_hw_machine(SYSCTL_HANDLER_ARGS) +{ +#ifdef SCTL_MASK32 + static const char machine32[] =3D "i386"; +#endif + int error; + +#ifdef SCTL_MASK32 + if ((req->flags & SCTL_MASK32) !=3D 0) + error =3D SYSCTL_OUT(req, machine32, sizeof(machine32)); + else +#endif + error =3D SYSCTL_OUT(req, machine, sizeof(machine)); + return (error); + +} +SYSCTL_PROC(_hw, HW_MACHINE, machine, CTLTYPE_STRING | CTLFLAG_RD, + NULL, 0, sysctl_hw_machine, "A", "Machine class"); =20 static char cpu_model[128]; SYSCTL_STRING(_hw, HW_MODEL, model, CTLFLAG_RD,=20 diff --git a/sys/kern/kern_mib.c b/sys/kern/kern_mib.c index 7ef580f..0b7d27f 100644 --- a/sys/kern/kern_mib.c +++ b/sys/kern/kern_mib.c @@ -232,9 +232,31 @@ sysctl_hw_pagesizes(SYSCTL_HANDLER_ARGS) SYSCTL_PROC(_hw, OID_AUTO, pagesizes, CTLTYPE_ULONG | CTLFLAG_RD, NULL, 0, sysctl_hw_pagesizes, "LU", "Supported page sizes"); =20 -static char machine_arch[] =3D MACHINE_ARCH; -SYSCTL_STRING(_hw, HW_MACHINE_ARCH, machine_arch, CTLFLAG_RD, - machine_arch, 0, "System architecture"); +static int +sysctl_hw_machine_arch(SYSCTL_HANDLER_ARGS) +{ + static const char machine_arch[] =3D MACHINE_ARCH; +#ifdef SCTL_MASK32 + static const char machine_arch32[] =3D=20 +#if defined(__amd64__) || defined(__ia64__) + "i386"; +#else + MACHINE_ARCH; +#endif +#endif + int error; + +#ifdef SCTL_MASK32 + if ((req->flags & SCTL_MASK32) !=3D 0) + error =3D SYSCTL_OUT(req, machine_arch32, sizeof(machine_arch32)); + else +#endif + error =3D SYSCTL_OUT(req, machine_arch, sizeof(machine_arch)); + return (error); + +} +SYSCTL_PROC(_hw, HW_MACHINE_ARCH, machine_arch, CTLTYPE_STRING | CTLFLAG_R= D, + NULL, 0, sysctl_hw_machine_arch, "A", "System architecture"); =20 static int sysctl_hostname(SYSCTL_HANDLER_ARGS) --Cjvo8w+V8lMs5Jsv Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkxExA4ACgkQC3+MBN1Mb4gU1wCeKOkfffKkS5WYTXHjuIdgO+V6 KB0AoI6WR+a6LS3/KA+SnQeULxS16EAX =AEoP -----END PGP SIGNATURE----- --Cjvo8w+V8lMs5Jsv-- From owner-freebsd-arch@FreeBSD.ORG Mon Jul 19 21:52:35 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A3FF1065670 for ; Mon, 19 Jul 2010 21:52:35 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 102F68FC19 for ; Mon, 19 Jul 2010 21:52:34 +0000 (UTC) Received: from [127.0.0.1] (pooker.samsco.org [168.103.85.57]) (authenticated bits=0) by pooker.samsco.org (8.14.4/8.14.4) with ESMTP id o6JLqVAv066358; Mon, 19 Jul 2010 15:52:31 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> Date: Mon, 19 Jul 2010 15:52:31 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> To: Kostik Belousov X-Mailer: Apple Mail (2.1078) X-Spam-Status: No, score=-50.0 required=3.8 tests=ALL_TRUSTED, T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: amd64@freebsd.org, arch@freebsd.org Subject: Re: uname -m/-p for compat32 binaries X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jul 2010 21:52:35 -0000 We do something similar at yahoo, and it's code that we're working on = packaging up to put back into FreeBSD. I don't know how your code = differs from ours, and I obviously cannot stop you from committing = yours, but you're welcome to look at our code. Scott On Jul 19, 2010, at 3:30 PM, Kostik Belousov wrote: > Hi, > I intend to commit the following change, that makes sysctls > hw.machine_arch and hw.machine to return "i386" for 32 bit > binaries run on amd64. In particular, 32 bit uname -m and uname -p > print "i386", that is good for i386 jails on amd64 kernels. >=20 > I find the change very useful for me, but I wonder why such trivial > modification is not yet done. Can anybody note a possible fallout from > it ? >=20 > diff --git a/sys/amd64/amd64/identcpu.c b/sys/amd64/amd64/identcpu.c > index 52e7568..68de22b 100644 > --- a/sys/amd64/amd64/identcpu.c > +++ b/sys/amd64/amd64/identcpu.c > @@ -76,8 +76,26 @@ static void print_via_padlock_info(void); >=20 > int cpu_class; > char machine[] =3D "amd64"; > -SYSCTL_STRING(_hw, HW_MACHINE, machine, CTLFLAG_RD,=20 > - machine, 0, "Machine class"); > + > +static int > +sysctl_hw_machine(SYSCTL_HANDLER_ARGS) > +{ > +#ifdef SCTL_MASK32 > + static const char machine32[] =3D "i386"; > +#endif > + int error; > + > +#ifdef SCTL_MASK32 > + if ((req->flags & SCTL_MASK32) !=3D 0) > + error =3D SYSCTL_OUT(req, machine32, sizeof(machine32)); > + else > +#endif > + error =3D SYSCTL_OUT(req, machine, sizeof(machine)); > + return (error); > + > +} > +SYSCTL_PROC(_hw, HW_MACHINE, machine, CTLTYPE_STRING | CTLFLAG_RD, > + NULL, 0, sysctl_hw_machine, "A", "Machine class"); >=20 > static char cpu_model[128]; > SYSCTL_STRING(_hw, HW_MODEL, model, CTLFLAG_RD,=20 > diff --git a/sys/kern/kern_mib.c b/sys/kern/kern_mib.c > index 7ef580f..0b7d27f 100644 > --- a/sys/kern/kern_mib.c > +++ b/sys/kern/kern_mib.c > @@ -232,9 +232,31 @@ sysctl_hw_pagesizes(SYSCTL_HANDLER_ARGS) > SYSCTL_PROC(_hw, OID_AUTO, pagesizes, CTLTYPE_ULONG | CTLFLAG_RD, > NULL, 0, sysctl_hw_pagesizes, "LU", "Supported page sizes"); >=20 > -static char machine_arch[] =3D MACHINE_ARCH; > -SYSCTL_STRING(_hw, HW_MACHINE_ARCH, machine_arch, CTLFLAG_RD, > - machine_arch, 0, "System architecture"); > +static int > +sysctl_hw_machine_arch(SYSCTL_HANDLER_ARGS) > +{ > + static const char machine_arch[] =3D MACHINE_ARCH; > +#ifdef SCTL_MASK32 > + static const char machine_arch32[] =3D=20 > +#if defined(__amd64__) || defined(__ia64__) > + "i386"; > +#else > + MACHINE_ARCH; > +#endif > +#endif > + int error; > + > +#ifdef SCTL_MASK32 > + if ((req->flags & SCTL_MASK32) !=3D 0) > + error =3D SYSCTL_OUT(req, machine_arch32, = sizeof(machine_arch32)); > + else > +#endif > + error =3D SYSCTL_OUT(req, machine_arch, = sizeof(machine_arch)); > + return (error); > + > +} > +SYSCTL_PROC(_hw, HW_MACHINE_ARCH, machine_arch, CTLTYPE_STRING | = CTLFLAG_RD, > + NULL, 0, sysctl_hw_machine_arch, "A", "System architecture"); >=20 > static int > sysctl_hostname(SYSCTL_HANDLER_ARGS) From owner-freebsd-arch@FreeBSD.ORG Mon Jul 19 21:57:52 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A47B91065670; Mon, 19 Jul 2010 21:57:52 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 2290C8FC19; Mon, 19 Jul 2010 21:57:51 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o6JLvmFb081109 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 20 Jul 2010 00:57:48 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o6JLvmA5072267; Tue, 20 Jul 2010 00:57:48 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o6JLvk0C072266; Tue, 20 Jul 2010 00:57:46 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 20 Jul 2010 00:57:46 +0300 From: Kostik Belousov To: Scott Long Message-ID: <20100719215746.GC2381@deviant.kiev.zoral.com.ua> References: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="UIrAl4r1g2eOkvhC" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.2 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_50, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: amd64@freebsd.org, arch@freebsd.org Subject: Re: uname -m/-p for compat32 binaries X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jul 2010 21:57:52 -0000 --UIrAl4r1g2eOkvhC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Jul 19, 2010 at 03:52:31PM -0600, Scott Long wrote: > We do something similar at yahoo, and it's code that we're working > on packaging up to put back into FreeBSD. I don't know how your code > differs from ours, and I obviously cannot stop you from committing > yours, but you're welcome to look at our code. There is obviously no rush to commit this snippet, and I obviously would abstain if this would make larger integration harder. Where to look ? Or should I just sit and wait ? --UIrAl4r1g2eOkvhC Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkxEylkACgkQC3+MBN1Mb4jJhgCgjTkOO/BiL6a6F9ninEwfP2QI AvEAoOAqVfpuOyzYIZyL0es/NaNFqpIU =t4xJ -----END PGP SIGNATURE----- --UIrAl4r1g2eOkvhC-- From owner-freebsd-arch@FreeBSD.ORG Mon Jul 19 22:17:20 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A0211065676; Mon, 19 Jul 2010 22:17:20 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 0D5458FC19; Mon, 19 Jul 2010 22:17:19 +0000 (UTC) Received: from [127.0.0.1] (pooker.samsco.org [168.103.85.57]) (authenticated bits=0) by pooker.samsco.org (8.14.4/8.14.4) with ESMTP id o6JMHGmc066479; Mon, 19 Jul 2010 16:17:16 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <20100719215746.GC2381@deviant.kiev.zoral.com.ua> Date: Mon, 19 Jul 2010 16:17:16 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> <20100719215746.GC2381@deviant.kiev.zoral.com.ua> To: Kostik Belousov X-Mailer: Apple Mail (2.1078) X-Spam-Status: No, score=-50.0 required=3.8 tests=ALL_TRUSTED, T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: amd64@freebsd.org, arch@freebsd.org Subject: Re: uname -m/-p for compat32 binaries X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jul 2010 22:17:20 -0000 Just checked, and I was a little off. We don't actually do this in the = kernel, we override it in the environment UNAME_ variables. All of our = software that wants to look at the machine arch uses uname to do it, so = we go that route. That way, we're not really lying to anything that = wants to get the definitive answer from the hw.machine architecture. I = can't defend it any further than that, maybe Peter or Paul or John can = comment on it. I personally don't see one way as being better than the = other, as they both have potential problems. As you noted in your = previous email, it's an easy change that could have been done long ago; = maybe the fact that it hasn't points to a good reason not to. Scott On Jul 19, 2010, at 3:57 PM, Kostik Belousov wrote: > On Mon, Jul 19, 2010 at 03:52:31PM -0600, Scott Long wrote: >> We do something similar at yahoo, and it's code that we're working >> on packaging up to put back into FreeBSD. I don't know how your code >> differs from ours, and I obviously cannot stop you from committing >> yours, but you're welcome to look at our code. > There is obviously no rush to commit this snippet, and I obviously = would > abstain if this would make larger integration harder. >=20 > Where to look ? Or should I just sit and wait ? From owner-freebsd-arch@FreeBSD.ORG Tue Jul 20 10:54:24 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C3711065673 for ; Tue, 20 Jul 2010 10:54:24 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 0DA378FC0C for ; Tue, 20 Jul 2010 10:54:23 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 3FBDC1FFC33; Tue, 20 Jul 2010 10:54:23 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 47E0E8457B; Tue, 20 Jul 2010 12:52:09 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Kostik Belousov References: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> Date: Tue, 20 Jul 2010 12:52:09 +0200 In-Reply-To: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> (Kostik Belousov's message of "Tue, 20 Jul 2010 00:30:54 +0300") Message-ID: <86eieyl92e.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: amd64@freebsd.org, arch@freebsd.org Subject: Re: uname -m/-p for compat32 binaries X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jul 2010 10:54:24 -0000 Kostik Belousov writes: > I intend to commit the following change, that makes sysctls > hw.machine_arch and hw.machine to return "i386" for 32 bit > binaries run on amd64. In particular, 32 bit uname -m and uname -p > print "i386", that is good for i386 jails on amd64 kernels. *thank you*, this trips me up every time I need to set up a 32-bit chroot or jail for some reason. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Tue Jul 20 11:48:55 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B9E56106566C; Tue, 20 Jul 2010 11:48:55 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 3DBD78FC14; Tue, 20 Jul 2010 11:48:54 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o6KBmp6K048237 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 20 Jul 2010 14:48:51 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o6KBmoEb087189; Tue, 20 Jul 2010 14:48:50 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o6KBmoH3087188; Tue, 20 Jul 2010 14:48:50 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 20 Jul 2010 14:48:50 +0300 From: Kostik Belousov To: Scott Long Message-ID: <20100720114850.GE2381@deviant.kiev.zoral.com.ua> References: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> <20100719215746.GC2381@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="NfRCDefLLAVdwZEC" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.2 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_50, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: amd64@freebsd.org, arch@freebsd.org Subject: Re: uname -m/-p for compat32 binaries X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jul 2010 11:48:55 -0000 --NfRCDefLLAVdwZEC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 19, 2010 at 04:17:16PM -0600, Scott Long wrote: > Just checked, and I was a little off. We don't actually do this in the > kernel, we override it in the environment UNAME_ variables. All of our > software that wants to look at the machine arch uses uname to do it, > so we go that route. That way, we're not really lying to anything that > wants to get the definitive answer from the hw.machine architecture. > I can't defend it any further than that, maybe Peter or Paul or John > can comment on it. I personally don't see one way as being better than > the other, as they both have potential problems. As you noted in your > previous email, it's an easy change that could have been done long ago; > maybe the fact that it hasn't points to a good reason not to. I know about environment variables affecting uname output, and use it if possible. Unfortunately, there are some situations where environment not propagated to the childs, or explicitely cleaned, e.g. sudo without -E. Or, it is hard to establish environment at the first place. I plan to commit it tomorrow. >=20 >=20 > Scott >=20 >=20 >=20 > On Jul 19, 2010, at 3:57 PM, Kostik Belousov wrote: >=20 > > On Mon, Jul 19, 2010 at 03:52:31PM -0600, Scott Long wrote: > >> We do something similar at yahoo, and it's code that we're working > >> on packaging up to put back into FreeBSD. I don't know how your code > >> differs from ours, and I obviously cannot stop you from committing > >> yours, but you're welcome to look at our code. > > There is obviously no rush to commit this snippet, and I obviously would > > abstain if this would make larger integration harder. > >=20 > > Where to look ? Or should I just sit and wait ? >=20 --NfRCDefLLAVdwZEC Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkxFjSEACgkQC3+MBN1Mb4iyCACgxnwhT9FJ7RCrhSY68etBSx3i NSIAnRz/niqDmFWHCOD9bo+WEF5J3CXj =vtyI -----END PGP SIGNATURE----- --NfRCDefLLAVdwZEC-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 20 14:43:44 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 24E2B106567D; Tue, 20 Jul 2010 14:43:44 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id EAB488FC31; Tue, 20 Jul 2010 14:43:43 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 84B3C46B37; Tue, 20 Jul 2010 10:43:43 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id B12308A03C; Tue, 20 Jul 2010 10:43:42 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org, amd64@freebsd.org Date: Tue, 20 Jul 2010 09:07:24 -0400 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20100217; KDE/4.4.5; amd64; ; ) References: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> In-Reply-To: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201007200907.24715.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 20 Jul 2010 10:43:42 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Kostik Belousov Subject: Re: uname -m/-p for compat32 binaries X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jul 2010 14:43:44 -0000 On Monday, July 19, 2010 5:30:54 pm Kostik Belousov wrote: > Hi, > I intend to commit the following change, that makes sysctls > hw.machine_arch and hw.machine to return "i386" for 32 bit > binaries run on amd64. In particular, 32 bit uname -m and uname -p > print "i386", that is good for i386 jails on amd64 kernels. > > I find the change very useful for me, but I wonder why such trivial > modification is not yet done. Can anybody note a possible fallout from > it ? Presumably ia64 and powerpc64 would need a similar change as well? It looks fine to me. I suspect Y! used the UNAME_* approach as it didn't add yet- another local diff to maintain in the kernel, and the uname fixes at Y! might have predated SCTL_MASK32. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Jul 20 17:51:29 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AC806106566C; Tue, 20 Jul 2010 17:51:29 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3CDE68FC17; Tue, 20 Jul 2010 17:51:29 +0000 (UTC) Received: by gxk24 with SMTP id 24so3761237gxk.13 for ; Tue, 20 Jul 2010 10:51:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=hziPvfyRd1ng0BjCPJVIPXK3XBtkxtO8ZQgYTtn2lgQ=; b=iGR5pGzKjSTcDaEF/ed3BUMneZPv/x4ISokbfKB7aoYkx0PzoI6l9OmCk2RmKKcMfY 2H9GBordPMfpLqaYbHFJGm/8xKTDbD4SY5Dsg/CYVNA9sSMH2OojNL8kwEFWfaFRnvs2 Mns045hXOMg3+7zL3m95/9+ALAHIolKi6oh0Y= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=EO/hx2/VgIw8tTs9iSTtI4P/zbDaCqP/8kDGbn/ySkdsyUDhA3HVwOnCoxLmmWB4fn vUJvEIx+KikereabKv8tW3JQ/9OZPgDH2QWhs45pL/ufP17jb71SkQUnp/2TOpP2dmqj XIXWyScqfAFTE415JV4AqDJVrWwMknPYRGE2Q= MIME-Version: 1.0 Received: by 10.150.214.11 with SMTP id m11mr698798ybg.315.1279648287133; Tue, 20 Jul 2010 10:51:27 -0700 (PDT) Sender: yanegomi@gmail.com Received: by 10.231.169.18 with HTTP; Tue, 20 Jul 2010 10:51:27 -0700 (PDT) In-Reply-To: <201007200907.24715.jhb@freebsd.org> References: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> <201007200907.24715.jhb@freebsd.org> Date: Tue, 20 Jul 2010 10:51:27 -0700 X-Google-Sender-Auth: rLpPLJW_ZxNxl4vYjyckGykUX0g Message-ID: From: Garrett Cooper To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: amd64@freebsd.org, Kostik Belousov , freebsd-arch@freebsd.org Subject: Re: uname -m/-p for compat32 binaries X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jul 2010 17:51:29 -0000 On Tue, Jul 20, 2010 at 6:07 AM, John Baldwin wrote: > On Monday, July 19, 2010 5:30:54 pm Kostik Belousov wrote: >> Hi, >> I intend to commit the following change, that makes sysctls >> hw.machine_arch and hw.machine to return "i386" for 32 bit >> binaries run on amd64. In particular, 32 bit uname -m and uname -p >> print "i386", that is good for i386 jails on amd64 kernels. >> >> I find the change very useful for me, but I wonder why such trivial >> modification is not yet done. Can anybody note a possible fallout from >> it ? > > Presumably ia64 and powerpc64 would need a similar change as well? =A0It = looks > fine to me. =A0I suspect Y! used the UNAME_* approach as it didn't add ye= t- > another local diff to maintain in the kernel, and the uname fixes at Y! m= ight > have predated SCTL_MASK32. I thought amd64 was a special case because we run biarch, whereas ia64 was always 64-bit... Wouldn't this change also create problems later on down the line when we're no longer biarch? Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Tue Jul 20 18:16:16 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 318E91065670; Tue, 20 Jul 2010 18:16:16 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 012B38FC08; Tue, 20 Jul 2010 18:16:16 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id A3EFB46B52; Tue, 20 Jul 2010 14:16:15 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id D2F888A052; Tue, 20 Jul 2010 14:16:14 -0400 (EDT) From: John Baldwin To: Garrett Cooper Date: Tue, 20 Jul 2010 14:15:14 -0400 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20100217; KDE/4.4.5; amd64; ; ) References: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> <201007200907.24715.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201007201415.14497.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 20 Jul 2010 14:16:14 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: amd64@freebsd.org, Kostik Belousov , freebsd-arch@freebsd.org Subject: Re: uname -m/-p for compat32 binaries X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jul 2010 18:16:16 -0000 On Tuesday, July 20, 2010 1:51:27 pm Garrett Cooper wrote: > On Tue, Jul 20, 2010 at 6:07 AM, John Baldwin wrote: > > On Monday, July 19, 2010 5:30:54 pm Kostik Belousov wrote: > >> Hi, > >> I intend to commit the following change, that makes sysctls > >> hw.machine_arch and hw.machine to return "i386" for 32 bit > >> binaries run on amd64. In particular, 32 bit uname -m and uname -p > >> print "i386", that is good for i386 jails on amd64 kernels. > >> > >> I find the change very useful for me, but I wonder why such trivial > >> modification is not yet done. Can anybody note a possible fallout from > >> it ? > > > > Presumably ia64 and powerpc64 would need a similar change as well? It looks > > fine to me. I suspect Y! used the UNAME_* approach as it didn't add yet- > > another local diff to maintain in the kernel, and the uname fixes at Y! might > > have predated SCTL_MASK32. > > I thought amd64 was a special case because we run biarch, whereas ia64 > was always 64-bit... > > Wouldn't this change also create problems later on down the line when > we're no longer biarch? amd64 is not biarch, it just supports FreeBSD/i386 binaries similar to how FreeBSD/i386 supports Linux/i386 binaries. Kostik's patch makes a FreeBSD/i386 uname binary report that it is running under FreeBSD/i386 just as we currently make a Linux/i386 uname binary report that it is running under Linux/i386. ia64 and powerpc64 both support COMPAT_FREEBSD32 similar to amd64. COMPAT_FREEBSD32 for amd64 and ia64 supports FreeBSD/i386 binaries. COMPAT_FREEBSD32 for powerpc64 supports FreeBSD/powerpc binaries. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Jul 20 20:49:13 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B96E1065673; Tue, 20 Jul 2010 20:49:13 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from mail.icecube.wisc.edu (trout.icecube.wisc.edu [128.104.255.119]) by mx1.freebsd.org (Postfix) with ESMTP id F1E558FC12; Tue, 20 Jul 2010 20:49:12 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.icecube.wisc.edu (Postfix) with ESMTP id 99629582CB; Tue, 20 Jul 2010 15:49:11 -0500 (CDT) X-Virus-Scanned: amavisd-new at icecube.wisc.edu Received: from mail.icecube.wisc.edu ([127.0.0.1]) by localhost (trout.icecube.wisc.edu [127.0.0.1]) (amavisd-new, port 10030) with ESMTP id LzFL7D12HjIb; Tue, 20 Jul 2010 15:49:11 -0500 (CDT) Received: from wanderer.tachypleus.net (mar92-14-88-173-196-182.fbx.proxad.net [88.173.196.182]) by mail.icecube.wisc.edu (Postfix) with ESMTP id 61990582C8; Tue, 20 Jul 2010 15:49:10 -0500 (CDT) Message-ID: <4C460BB9.1060009@freebsd.org> Date: Tue, 20 Jul 2010 15:48:57 -0500 From: Nathan Whitehorn User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.10) Gecko/20100627 Thunderbird/3.0.5 MIME-Version: 1.0 To: John Baldwin References: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> <201007200907.24715.jhb@freebsd.org> In-Reply-To: <201007200907.24715.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit Cc: amd64@freebsd.org, Kostik Belousov , freebsd-arch@freebsd.org Subject: Re: uname -m/-p for compat32 binaries X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jul 2010 20:49:13 -0000 On 07/20/10 08:07, John Baldwin wrote: > On Monday, July 19, 2010 5:30:54 pm Kostik Belousov wrote: > >> Hi, >> I intend to commit the following change, that makes sysctls >> hw.machine_arch and hw.machine to return "i386" for 32 bit >> binaries run on amd64. In particular, 32 bit uname -m and uname -p >> print "i386", that is good for i386 jails on amd64 kernels. >> >> I find the change very useful for me, but I wonder why such trivial >> modification is not yet done. Can anybody note a possible fallout from >> it ? >> > Presumably ia64 and powerpc64 would need a similar change as well? It looks > fine to me. I suspect Y! used the UNAME_* approach as it didn't add yet- > another local diff to maintain in the kernel, and the uname fixes at Y! might > have predated SCTL_MASK32. > Maybe it makes sense to define a MACHINE_ARCH32 in machine/param.h, as is done for ELF_ARCH32 in machine/elf.h? This would keep the MI code in the kernel MI, and ever-so-slightly simplify implementation for ia64, mips, and powerpc64. Thanks for doing this! -Nathan From owner-freebsd-arch@FreeBSD.ORG Wed Jul 21 12:00:33 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8F0281065690; Wed, 21 Jul 2010 12:00:33 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 2404A8FC18; Wed, 21 Jul 2010 12:00:32 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o6LC0TJ7082533 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 21 Jul 2010 15:00:29 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o6LC0Tnx045562; Wed, 21 Jul 2010 15:00:29 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o6LC0SFH045561; Wed, 21 Jul 2010 15:00:28 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 21 Jul 2010 15:00:28 +0300 From: Kostik Belousov To: Nathan Whitehorn Message-ID: <20100721120028.GM2381@deviant.kiev.zoral.com.ua> References: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> <201007200907.24715.jhb@freebsd.org> <4C460BB9.1060009@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="anmYPbAsEPjx7802" Content-Disposition: inline In-Reply-To: <4C460BB9.1060009@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.2 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_50, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: amd64@freebsd.org, freebsd-arch@freebsd.org Subject: Re: uname -m/-p for compat32 binaries X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jul 2010 12:00:33 -0000 --anmYPbAsEPjx7802 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 20, 2010 at 03:48:57PM -0500, Nathan Whitehorn wrote: > On 07/20/10 08:07, John Baldwin wrote: > >On Monday, July 19, 2010 5:30:54 pm Kostik Belousov wrote: > > =20 > >>Hi, > >>I intend to commit the following change, that makes sysctls > >>hw.machine_arch and hw.machine to return "i386" for 32 bit > >>binaries run on amd64. In particular, 32 bit uname -m and uname -p > >>print "i386", that is good for i386 jails on amd64 kernels. > >> > >>I find the change very useful for me, but I wonder why such trivial > >>modification is not yet done. Can anybody note a possible fallout from > >>it ? > >> =20 > >Presumably ia64 and powerpc64 would need a similar change as well? It= =20 > >looks > >fine to me. I suspect Y! used the UNAME_* approach as it didn't add yet- > >another local diff to maintain in the kernel, and the uname fixes at Y!= =20 > >might > >have predated SCTL_MASK32. > > =20 > Maybe it makes sense to define a MACHINE_ARCH32 in machine/param.h, as=20 > is done for ELF_ARCH32 in machine/elf.h? This would keep the MI code in= =20 > the kernel MI, and ever-so-slightly simplify implementation for ia64,=20 > mips, and powerpc64. Thanks for doing this! See below. Also, I painted red another wall, adding a sysctl to turn the adaptive behaviour off. diff --git a/sys/amd64/amd64/identcpu.c b/sys/amd64/amd64/identcpu.c index 52e7568..914f4d2 100644 --- a/sys/amd64/amd64/identcpu.c +++ b/sys/amd64/amd64/identcpu.c @@ -76,8 +76,28 @@ static void print_via_padlock_info(void); =20 int cpu_class; char machine[] =3D "amd64"; -SYSCTL_STRING(_hw, HW_MACHINE, machine, CTLFLAG_RD,=20 - machine, 0, "Machine class"); + +extern int adaptive_machine_arch; + +static int +sysctl_hw_machine(SYSCTL_HANDLER_ARGS) +{ +#ifdef SCTL_MASK32 + static const char machine32[] =3D "i386"; +#endif + int error; + +#ifdef SCTL_MASK32 + if ((req->flags & SCTL_MASK32) !=3D 0 && adaptive_machine_arch) + error =3D SYSCTL_OUT(req, machine32, sizeof(machine32)); + else +#endif + error =3D SYSCTL_OUT(req, machine, sizeof(machine)); + return (error); + +} +SYSCTL_PROC(_hw, HW_MACHINE, machine, CTLTYPE_STRING | CTLFLAG_RD, + NULL, 0, sysctl_hw_machine, "A", "Machine class"); =20 static char cpu_model[128]; SYSCTL_STRING(_hw, HW_MODEL, model, CTLFLAG_RD,=20 diff --git a/sys/amd64/include/param.h b/sys/amd64/include/param.h index c940597..9a742f9 100644 --- a/sys/amd64/include/param.h +++ b/sys/amd64/include/param.h @@ -59,6 +59,9 @@ #ifndef MACHINE_ARCH #define MACHINE_ARCH "amd64" #endif +#ifndef MACHINE_ARCH32 +#define MACHINE_ARCH32 "i386" +#endif =20 #if defined(SMP) || defined(KLD_MODULE) #define MAXCPU 32 diff --git a/sys/ia64/include/param.h b/sys/ia64/include/param.h index ba26290..36b27e0 100644 --- a/sys/ia64/include/param.h +++ b/sys/ia64/include/param.h @@ -57,6 +57,9 @@ #ifndef MACHINE_ARCH #define MACHINE_ARCH "ia64" #endif +#ifndef MACHINE_ARCH32 +#define MACHINE_ARCH32 "i386" +#endif =20 #if defined(SMP) || defined(KLD_MODULE) #define MAXCPU 32 diff --git a/sys/kern/kern_mib.c b/sys/kern/kern_mib.c index 7ef580f..3bfecea 100644 --- a/sys/kern/kern_mib.c +++ b/sys/kern/kern_mib.c @@ -232,9 +232,28 @@ sysctl_hw_pagesizes(SYSCTL_HANDLER_ARGS) SYSCTL_PROC(_hw, OID_AUTO, pagesizes, CTLTYPE_ULONG | CTLFLAG_RD, NULL, 0, sysctl_hw_pagesizes, "LU", "Supported page sizes"); =20 -static char machine_arch[] =3D MACHINE_ARCH; -SYSCTL_STRING(_hw, HW_MACHINE_ARCH, machine_arch, CTLFLAG_RD, - machine_arch, 0, "System architecture"); +int adaptive_machine_arch =3D 1; +static int +sysctl_hw_machine_arch(SYSCTL_HANDLER_ARGS) +{ + int error; + static const char machine_arch[] =3D MACHINE_ARCH; +#ifdef SCTL_MASK32 + static const char machine_arch32[] =3D MACHINE_ARCH32; + + if ((req->flags & SCTL_MASK32) !=3D 0 && adaptive_machine_arch) + error =3D SYSCTL_OUT(req, machine_arch32, sizeof(machine_arch32)); + else +#endif + error =3D SYSCTL_OUT(req, machine_arch, sizeof(machine_arch)); + return (error); + +} +SYSCTL_PROC(_hw, HW_MACHINE_ARCH, machine_arch, CTLTYPE_STRING | CTLFLAG_R= D, + NULL, 0, sysctl_hw_machine_arch, "A", "System architecture"); +SYSCTL_INT(_debug, OID_AUTO, adaptive_machine_arch, CTLFLAG_RW, + &adaptive_machine_arch, 1, + "Adapt reported machine architecture to the ABI of the binary"); =20 static int sysctl_hostname(SYSCTL_HANDLER_ARGS) diff --git a/sys/powerpc/include/param.h b/sys/powerpc/include/param.h index 4aeabe7..91bb238 100644 --- a/sys/powerpc/include/param.h +++ b/sys/powerpc/include/param.h @@ -61,6 +61,11 @@ #endif #endif #define MID_MACHINE MID_POWERPC +#ifdef __powerpc64__ +#ifndef MACHINE_ARCH32 +#define MACHINE_ARCH32 "powerpc" +#endif +#endif =20 #if defined(SMP) || defined(KLD_MODULE) #define MAXCPU 2 --anmYPbAsEPjx7802 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkxG4VwACgkQC3+MBN1Mb4hbGgCZAWr8jbD9YJ30/gAjd7up8ShA J6cAnROwLg7mlwmNpMjftFBgj/1UkYAN =4Gq8 -----END PGP SIGNATURE----- --anmYPbAsEPjx7802-- From owner-freebsd-arch@FreeBSD.ORG Wed Jul 21 21:30:18 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 20D7A1065677; Wed, 21 Jul 2010 21:30:18 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from mail.icecube.wisc.edu (trout.icecube.wisc.edu [128.104.255.119]) by mx1.freebsd.org (Postfix) with ESMTP id D2DF68FC13; Wed, 21 Jul 2010 21:30:17 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.icecube.wisc.edu (Postfix) with ESMTP id CE870582C9; Wed, 21 Jul 2010 16:30:16 -0500 (CDT) X-Virus-Scanned: amavisd-new at icecube.wisc.edu Received: from mail.icecube.wisc.edu ([127.0.0.1]) by localhost (trout.icecube.wisc.edu [127.0.0.1]) (amavisd-new, port 10030) with ESMTP id nwSzQnmcP3xj; Wed, 21 Jul 2010 16:30:16 -0500 (CDT) Received: from wanderer.tachypleus.net (mar92-14-88-173-196-182.fbx.proxad.net [88.173.196.182]) by mail.icecube.wisc.edu (Postfix) with ESMTP id 8AFE8582C8; Wed, 21 Jul 2010 16:30:15 -0500 (CDT) Message-ID: <4C4766DA.3020807@freebsd.org> Date: Wed, 21 Jul 2010 16:30:02 -0500 From: Nathan Whitehorn User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.10) Gecko/20100627 Thunderbird/3.0.5 MIME-Version: 1.0 To: Kostik Belousov References: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> <201007200907.24715.jhb@freebsd.org> <4C460BB9.1060009@freebsd.org> <20100721120028.GM2381@deviant.kiev.zoral.com.ua> In-Reply-To: <20100721120028.GM2381@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: amd64@freebsd.org, freebsd-arch@freebsd.org Subject: Re: uname -m/-p for compat32 binaries X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jul 2010 21:30:18 -0000 On 07/21/10 07:00, Kostik Belousov wrote: > On Tue, Jul 20, 2010 at 03:48:57PM -0500, Nathan Whitehorn wrote: > >> On 07/20/10 08:07, John Baldwin wrote: >> >>> On Monday, July 19, 2010 5:30:54 pm Kostik Belousov wrote: >>> >>> >>>> Hi, >>>> I intend to commit the following change, that makes sysctls >>>> hw.machine_arch and hw.machine to return "i386" for 32 bit >>>> binaries run on amd64. In particular, 32 bit uname -m and uname -p >>>> print "i386", that is good for i386 jails on amd64 kernels. >>>> >>>> I find the change very useful for me, but I wonder why such trivial >>>> modification is not yet done. Can anybody note a possible fallout from >>>> it ? >>>> >>>> >>> Presumably ia64 and powerpc64 would need a similar change as well? It >>> looks >>> fine to me. I suspect Y! used the UNAME_* approach as it didn't add yet- >>> another local diff to maintain in the kernel, and the uname fixes at Y! >>> might >>> have predated SCTL_MASK32. >>> >>> >> Maybe it makes sense to define a MACHINE_ARCH32 in machine/param.h, as >> is done for ELF_ARCH32 in machine/elf.h? This would keep the MI code in >> the kernel MI, and ever-so-slightly simplify implementation for ia64, >> mips, and powerpc64. Thanks for doing this! >> > See below. Also, I painted red another wall, adding a sysctl to turn > the adaptive behaviour off. > > I quite like the shade you've chosen! -Nathan From owner-freebsd-arch@FreeBSD.ORG Thu Jul 22 16:55:07 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C4C89106564A for ; Thu, 22 Jul 2010 16:55:07 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9C5828FC1C for ; Thu, 22 Jul 2010 16:55:07 +0000 (UTC) Received: by pvh1 with SMTP id 1so3624057pvh.13 for ; Thu, 22 Jul 2010 09:55:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=Hld6M2bRTI7+cCWo8SYf4ZuFK1h6p73gdGn5t4hpkE0=; b=Euo07CERtmaRXcMBK8Sm0DAd61H4L2tIu68m1l4FesSxoXI0rnHRUUQAD0vyDKCGkb J5+6XqnqoWC54rgwb6P7ap3eOKTV1jWGTc0PXJRwonAoYfkiiFcmiImAeXdyqZ64SQ9s RXlU13LI8ypoCL6M/pOc5hgrXriL96XzH/pNc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=jyOorcvG/VWnJ6SQBFWDwMITHfOHj7B0EY/hMvHKzxf5SD/BGpjT4E2kmdAJOEMgva pJ/UbWb9pleCFPZL9V0dVaPFHQRgS1d7Oq0zxi6iNwm3KqCXKBA21UeCOdklxX+SQVNI cv8Q8U7eubr8VaerKGit2F+JNHipQ7Btgk910= MIME-Version: 1.0 Received: by 10.114.107.6 with SMTP id f6mr3077033wac.54.1279817691803; Thu, 22 Jul 2010 09:54:51 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.42.6.85 with HTTP; Thu, 22 Jul 2010 09:54:51 -0700 (PDT) Date: Thu, 22 Jul 2010 09:54:51 -0700 X-Google-Sender-Auth: TRYyaztcjrEMqqaZy9wbWlsw5oA Message-ID: From: mdf@FreeBSD.org To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Multi-zone malloc(9) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 16:55:07 -0000 Occasionally we run into use-after-free and malloc'd buffer overrun scenarios. When this happens it can be rather difficult to determine what code is at fault, since e.g. every 64 byte allocation, regardless of malloc type, comes from the same UMA zone. This means that an overflow in M_TEMP will affect M_DEVBUF, etc. Adding multiple uma zones for each bucket size means that we can hash on the malloc type's shortdesc field so that there are fewer collisions and misused memory from one malloc type only affects a subset of other malloc types. Varying the hash means that, with several crashes due to memory stomp, a single malloc type can usually be determined as the culprit. If the bug isn't obvious from inspection at this point, MemGuard will help catch the offender. The patch at: http://people.freebsd.org/~mdf/multizone_malloc.patch implements an optional multi-zone malloc(9). By default there is a single zone, and MALLOC_DEBUG_MAXZONES can be specified in the kernel configuration file. A ddb function will print all the malloc types that have a hash collision with the specified type. A few questions for -arch@: - We found this very useful at Isilon. Should this go into CURRENT? - Should this be on by default for GENERIC? The memory overhead of 8 uma zones per malloc allocation size shouldn't be very large. - would a __FreeBSD_version bump be needed since the malloc_internal type is known by user-space? Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Thu Jul 22 17:41:25 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09D221065675; Thu, 22 Jul 2010 17:41:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 7C4358FC1F; Thu, 22 Jul 2010 17:41:24 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o6MHfKA9026825 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 22 Jul 2010 20:41:20 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o6MHfKAE027026; Thu, 22 Jul 2010 20:41:20 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o6MHfKEd027025; Thu, 22 Jul 2010 20:41:20 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 22 Jul 2010 20:41:20 +0300 From: Kostik Belousov To: mdf@freebsd.org Message-ID: <20100722174120.GR2381@deviant.kiev.zoral.com.ua> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ngluH3DisTFy0Cjl" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.2 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_50, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-arch@freebsd.org Subject: Re: Multi-zone malloc(9) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 17:41:25 -0000 --ngluH3DisTFy0Cjl Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jul 22, 2010 at 09:54:51AM -0700, mdf@freebsd.org wrote: > Occasionally we run into use-after-free and malloc'd buffer overrun > scenarios. When this happens it can be rather difficult to determine > what code is at fault, since e.g. every 64 byte allocation, regardless > of malloc type, comes from the same UMA zone. This means that an > overflow in M_TEMP will affect M_DEVBUF, etc. Adding multiple uma > zones for each bucket size means that we can hash on the malloc type's > shortdesc field so that there are fewer collisions and misused memory > from one malloc type only affects a subset of other malloc types. > Varying the hash means that, with several crashes due to memory stomp, > a single malloc type can usually be determined as the culprit. If the > bug isn't obvious from inspection at this point, MemGuard will help > catch the offender. >=20 > The patch at: >=20 > http://people.freebsd.org/~mdf/multizone_malloc.patch >=20 > implements an optional multi-zone malloc(9). By default there is a > single zone, and MALLOC_DEBUG_MAXZONES can be specified in the kernel > configuration file. A ddb function will print all the malloc types > that have a hash collision with the specified type. >=20 > A few questions for -arch@: >=20 > - We found this very useful at Isilon. Should this go into CURRENT? >=20 > - Should this be on by default for GENERIC? The memory overhead of 8 > uma zones per malloc allocation size shouldn't be very large. >=20 > - would a __FreeBSD_version bump be needed since the malloc_internal > type is known by user-space? Can you quantify the overhead, both in CPU time and memory usage terms ? I would much prefer to have debug and non-debug kernels to run similar code, in other words, can the multizone allocation be enabled unconditionally ? --ngluH3DisTFy0Cjl Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEUEARECAAYFAkxIgr8ACgkQC3+MBN1Mb4j2cwCfSRtigwr3xkLvmJ0eAbh4ho2y +70Al1bwaSE6Q4RtO+BaHj0hT6X8oRc= =mVa3 -----END PGP SIGNATURE----- --ngluH3DisTFy0Cjl-- From owner-freebsd-arch@FreeBSD.ORG Thu Jul 22 17:59:17 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6115D1065674 for ; Thu, 22 Jul 2010 17:59:17 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id E32F28FC08 for ; Thu, 22 Jul 2010 17:59:16 +0000 (UTC) Received: by ewy26 with SMTP id 26so3458183ewy.13 for ; Thu, 22 Jul 2010 10:59:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=X0vHkpvMe+udvrROZUJbxUGzy4Iu4kk/q0Jk1+Nd7F0=; b=kyU4qlDGWPRvBwdhDVyr9QM2q4+Ys+v8atbrTn7DaHjISfKawND0F5Kqht0R90kptP VYdM3eQGy06veN0memp0VYmfBGn/FFjGaLPr5wYpYVv5B0JrkTP0/Q+uSPd6mdfpPeLN +3lHZTNPnToOq+JDTtpf7uUyrOodKHjHeWuYs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=x3yrzv+tRJHbvWl40FA0J4kuGm+3+0L9P9dovRr9MH/DobMT/d1Y8GOCLyCKklEU8j ZauClnQsDD6u3ybnROSdn5sRX2FCaY/tfU5FqNuw+VxPD1YWtJGAH8kxdZroeZCYYMY+ 1Q8MRShJ+fHODwCmG44IjO6XQs8GE8PgW5/lk= MIME-Version: 1.0 Received: by 10.213.19.207 with SMTP id c15mr8399804ebb.1.1279821555302; Thu, 22 Jul 2010 10:59:15 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.42.6.85 with HTTP; Thu, 22 Jul 2010 10:59:14 -0700 (PDT) In-Reply-To: <20100722174120.GR2381@deviant.kiev.zoral.com.ua> References: <20100722174120.GR2381@deviant.kiev.zoral.com.ua> Date: Thu, 22 Jul 2010 10:59:14 -0700 X-Google-Sender-Auth: Wcexl4YW7dz2u-3CF6I1HMuImiU Message-ID: From: mdf@FreeBSD.org To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: Multi-zone malloc(9) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jul 2010 17:59:17 -0000 On Thu, Jul 22, 2010 at 10:41 AM, Kostik Belousov wro= te: > On Thu, Jul 22, 2010 at 09:54:51AM -0700, mdf@freebsd.org wrote: >> Occasionally we run into use-after-free and malloc'd buffer overrun >> scenarios. =A0When this happens it can be rather difficult to determine >> what code is at fault, since e.g. every 64 byte allocation, regardless >> of malloc type, comes from the same UMA zone. =A0This means that an >> overflow in M_TEMP will affect M_DEVBUF, etc. =A0Adding multiple uma >> zones for each bucket size means that we can hash on the malloc type's >> shortdesc field so that there are fewer collisions and misused memory >> from one malloc type only affects a subset of other malloc types. >> Varying the hash means that, with several crashes due to memory stomp, >> a single malloc type can usually be determined as the culprit. =A0If the >> bug isn't obvious from inspection at this point, MemGuard will help >> catch the offender. >> >> The patch at: >> >> =A0 =A0http://people.freebsd.org/~mdf/multizone_malloc.patch >> >> implements an optional multi-zone malloc(9). =A0By default there is a >> single zone, and MALLOC_DEBUG_MAXZONES can be specified in the kernel >> configuration file. =A0A ddb function will print all the malloc types >> that have a hash collision with the specified type. >> >> A few questions for -arch@: >> >> =A0- We found this very useful at Isilon. =A0Should this go into CURRENT= ? >> >> =A0- Should this be on by default for GENERIC? =A0The memory overhead of= 8 >> uma zones per malloc allocation size shouldn't be very large. >> >> =A0- would a __FreeBSD_version bump be needed since the malloc_internal >> type is known by user-space? > > Can you quantify the overhead, both in CPU time and memory usage terms > ? I would much prefer to have debug and non-debug kernels to run > similar code, in other words, can the multizone allocation be enabled > unconditionally ? CPU usage should be lost in the noise, since the subzone hash class is computed once at boot. Memory overhead is a bit hard to quantify, since with multiple zones there are more partially allocated pages in each malloc size. Worst case would be, I think, 1 page per CPU per multi-zone per allocation size, so for 4 CPUs with 4k pages, that would be 4*8*9 or a little over 1MB. The actual number of multi-zones is also a TUNABLE, so the same code could be built for debug and non-debug, and a loader.conf change would enable the actual use of multiple zones during boot. Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Fri Jul 23 13:47:30 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 602171065672 for ; Fri, 23 Jul 2010 13:47:30 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 1AA258FC08 for ; Fri, 23 Jul 2010 13:47:29 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1OcIb0-0004N4-GK for freebsd-arch@freebsd.org; Fri, 23 Jul 2010 15:47:26 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 23 Jul 2010 15:47:26 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 23 Jul 2010 15:47:26 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Fri, 23 Jul 2010 15:47:41 +0200 Lines: 19 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100518 Thunderbird/3.0.4 In-Reply-To: X-Enigmail-Version: 1.0.1 Subject: Re: Multi-zone malloc(9) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jul 2010 13:47:30 -0000 On 07/22/10 18:54, mdf@FreeBSD.org wrote: > Occasionally we run into use-after-free and malloc'd buffer overrun > scenarios. When this happens it can be rather difficult to determine > what code is at fault, since e.g. every 64 byte allocation, regardless > of malloc type, comes from the same UMA zone. This means that an > overflow in M_TEMP will affect M_DEVBUF, etc. Adding multiple uma > zones for each bucket size means that we can hash on the malloc type's > shortdesc field so that there are fewer collisions and misused memory > from one malloc type only affects a subset of other malloc types. To what extent does something like this help? As I read it, you still have the problem of overflows from one allocation trashing data in some other random allocation, but now you also have to track which hash bucket is it in while debugging? And would this interfere with possible NUMA efforts? (because it sort of looks similar - hashing allocations to "zones", which in NUMA case would be per-CPU). From owner-freebsd-arch@FreeBSD.ORG Fri Jul 23 15:48:28 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9F4C1065679 for ; Fri, 23 Jul 2010 15:48:28 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 7EF528FC14 for ; Fri, 23 Jul 2010 15:48:28 +0000 (UTC) Received: by yxe42 with SMTP id 42so3959388yxe.13 for ; Fri, 23 Jul 2010 08:48:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=ij4tMd8PQ89zDc5bSnfRwmXay8F5vHHn9HEHQsdd2dc=; b=XjHWDXULhD+i2jigojcTeFVyC2lBXf7cD6AQ3OE9d3AP0JxBcOezCAYdceRmvDd+cP Cad9FBGpVg8mS0q1E/Nl9TiUcciKd79Ui2nGogkJcQsgayaiImgUfeZ5DOkIPGq+ogBI 66RmvDiXEs0U1NGDmmaFPIQGUEp72l0U9oKSU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=IdPv/+rOA5JIUH+99QAHRhjmk9Dc9KKcaU+jB5xXa2r4eys8vu/l02NS1t4xl9XmTp dYChj2IHMB+p1Bu9IURRvb8c6cuS37sGxR0cgCvBn+a8weEJv8C2XXcpagy7BLV7xHFC ULGDaSb/AUFeTpgevK27vHai8EjBZ4vKKmNaE= MIME-Version: 1.0 Received: by 10.151.139.16 with SMTP id r16mr5916863ybn.168.1279900105708; Fri, 23 Jul 2010 08:48:25 -0700 (PDT) Received: by 10.42.6.85 with HTTP; Fri, 23 Jul 2010 08:48:25 -0700 (PDT) In-Reply-To: References: Date: Fri, 23 Jul 2010 08:48:25 -0700 Message-ID: From: Matthew Fleming To: Ivan Voras Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org Subject: Re: Multi-zone malloc(9) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jul 2010 15:48:28 -0000 On Fri, Jul 23, 2010 at 6:47 AM, Ivan Voras wrote: > On 07/22/10 18:54, mdf@FreeBSD.org wrote: >> Occasionally we run into use-after-free and malloc'd buffer overrun >> scenarios. =A0When this happens it can be rather difficult to determine >> what code is at fault, since e.g. every 64 byte allocation, regardless >> of malloc type, comes from the same UMA zone. =A0This means that an >> overflow in M_TEMP will affect M_DEVBUF, etc. =A0Adding multiple uma >> zones for each bucket size means that we can hash on the malloc type's >> shortdesc field so that there are fewer collisions and misused memory >> from one malloc type only affects a subset of other malloc types. > > To what extent does something like this help? As I read it, you still > have the problem of overflows from one allocation trashing data in some > other random allocation, but now you also have to track which hash > bucket is it in while debugging? It helps in that, with several repros on different hash functions, you now can know exactly which malloc type was doing the trashing, because it's the intersection of all the hash classes from each instance. Assuming you have a core dump, the hash class is listed in the malloc_type_internal hanging off the malloc type, so it's easy to correlate. There's a ddb function in the patch to print all the matching types, and a gdb script would be easy to write as well. > And would this interfere with possible NUMA efforts? (because it sort of > looks similar - hashing allocations to "zones", which in NUMA case would > be per-CPU). The NUMA effort with malloc/uma is still theoretical so I can't say for sure, but I believe these two issues are orthogonal. Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Fri Jul 23 18:31:26 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57F821065675; Fri, 23 Jul 2010 18:31:26 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 1BCA08FC0C; Fri, 23 Jul 2010 18:31:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o6NIPilF003315; Fri, 23 Jul 2010 12:25:45 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Fri, 23 Jul 2010 12:26:18 -0600 (MDT) Message-Id: <20100723.122618.750582998550767254.imp@bsdimp.com> To: jhb@FreeBSD.org From: "M. Warner Losh" In-Reply-To: <201007200907.24715.jhb@freebsd.org> References: <20100719213054.GB2381@deviant.kiev.zoral.com.ua> <201007200907.24715.jhb@freebsd.org> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: amd64@FreeBSD.org, kostikbel@gmail.com, freebsd-arch@FreeBSD.org Subject: Re: uname -m/-p for compat32 binaries X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jul 2010 18:31:26 -0000 In message: <201007200907.24715.jhb@freebsd.org> John Baldwin writes: : On Monday, July 19, 2010 5:30:54 pm Kostik Belousov wrote: : > Hi, : > I intend to commit the following change, that makes sysctls : > hw.machine_arch and hw.machine to return "i386" for 32 bit : > binaries run on amd64. In particular, 32 bit uname -m and uname -p : > print "i386", that is good for i386 jails on amd64 kernels. : > : > I find the change very useful for me, but I wonder why such trivial : > modification is not yet done. Can anybody note a possible fallout from : > it ? : : Presumably ia64 and powerpc64 would need a similar change as well? It looks : fine to me. I suspect Y! used the UNAME_* approach as it didn't add yet- : another local diff to maintain in the kernel, and the uname fixes at Y! might : have predated SCTL_MASK32. mips might as well, someday. Warner