From owner-freebsd-arch@FreeBSD.ORG Sun Jan 20 04:11:23 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3BEBE317 for ; Sun, 20 Jan 2013 04:11:23 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) by mx1.freebsd.org (Postfix) with ESMTP id 02359F45 for ; Sun, 20 Jan 2013 04:11:22 +0000 (UTC) Received: from Xins-MacBook-Pro-2.local (c-67-188-85-47.hsd1.ca.comcast.net [67.188.85.47]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by anubis.delphij.net (Postfix) with ESMTPSA id 65B03236D1; Sat, 19 Jan 2013 20:11:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=delphij.net; s=anubis; t=1358655076; bh=Y/wXHRNJ3TmlWir6YjUBsE5KsoZifALSDSqzZiBnbr0=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=0hn4YBDU/75rr2pdt4DAcdY3cTIZFgh3zHKRKsEXvqvVVT3JYY1uKGY3YedVUCzdx 0LSWlS2+CE7ponISW8N5cney/PwH/FRUQjFigkh2hC987YITm0Gu5DvdhzsE60d75/ IJt9uti4aQcbQ1jtgKSZI5l20dFLmcgbxGVCEl7I= Message-ID: <50FB6E64.6060203@delphij.net> Date: Sat, 19 Jan 2013 20:11:16 -0800 From: Xin Li Organization: The FreeBSD Project MIME-Version: 1.0 To: Chagin Dmitry Subject: Re: RFC: futimens(2) and utimensat(2) References: <4F4DC876.3010809@delphij.net> <20130119145929.GA23230@dchagin.static.corbina.net> In-Reply-To: <20130119145929.GA23230@dchagin.static.corbina.net> X-Enigmail-Version: 1.5 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: d@delphij.net, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: d@delphij.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jan 2013 04:11:23 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 1/19/13 6:59 AM, Chagin Dmitry wrote: > On Tue, Feb 28, 2012 at 10:40:54PM -0800, Xin Li wrote: >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 >> >> Hi, >> >> These are required by IEEE Std 1003.1-2008. Patchset at: >> >> http://people.freebsd.org/~delphij/for_review/utimens.diff >> >> Cheers, - -- Xin LI >> https://www.delphij.net/ FreeBSD - The Power to Serve! Live free >> or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 >> (FreeBSD) > > > Hi Xin, Do you plan to commit this? I'm not working on this recently, and as Jilles already pointed out, it needs to be improved or rewritten to make sure that its standard conformance. Do you plan to work on this or just asking? If you have time and plan to do it, be my guest. Cheers, -----BEGIN PGP SIGNATURE----- iQEcBAEBCAAGBQJQ+25kAAoJEG80Jeu8UPuz94cH/jpNvWpd5aLWgkx3yA1m7yGj jWOQadYV58TK6JIQn+gLd6mEU8IXOu74gBLCpQqdL4U9NiU4PhZ5sTfKXwDR7A2w uZUlQle2cfWLXKBVQI5vsHT0gdGyQ1HHtUFeIA6adSSKSpsM1JOmxMTU4e/f2tPQ s9CEdHcBEnSTLfhsFkoFO58N1sgLvFRGzh6EUWkAt9I8q5IW7enaZnI7sLdwRVkk IoOkKs35Sm7rag+IaJoegSxOs0w9WUZJLtoBPyKHFIihImt+6hBHNmIx7+Nz4Eih P4lwMU7wT/MobQxWeyN0dgZoA8Mj3VnjE972D2cllkeo7DfuYqE7jFEsaS08O2k= =WhU0 -----END PGP SIGNATURE----- From owner-freebsd-arch@FreeBSD.ORG Sun Jan 20 05:00:43 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 6E101B0C for ; Sun, 20 Jan 2013 05:00:43 +0000 (UTC) (envelope-from chagin.dmitry@gmail.com) Received: from mail-la0-f47.google.com (mail-la0-f47.google.com [209.85.215.47]) by mx1.freebsd.org (Postfix) with ESMTP id CD3B671 for ; Sun, 20 Jan 2013 05:00:42 +0000 (UTC) Received: by mail-la0-f47.google.com with SMTP id fh20so5119517lab.34 for ; Sat, 19 Jan 2013 21:00:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:date:from:to:cc:subject:message-id:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=iSnv+R8AEpADgn3O7ZIqA2lRUChBNi5FRepUX2Hoork=; b=t+FnR9ugdOXjbIELTt4GUN1nE8NdTUy+0rlmjYTNwP0gi5AZ7t4k0BN20IUcSGsb0G 8phae3LsewMFYqAOJhYwXX7pQAwUNG3uR1mb689sulp0lgSj3u5UIqqn6x232oWzvise c8gBzL0Sxd1CuGU9nx9b3bV9W5lpkDDY4QdlAW3fXVI64e4IVHKUJLN4A4SURILG7U0D QMv+BI/wa+KdWQafMB439aegrbXzCupjQb3olWJnQaC1ieVhp2Jhd8VHfAxqLRXSiGxl U+tIm+lOfetGVsMnkQ8BPPFpSmljFuhQXMi8RBTacbm+UYOZ/QL62M1TljOetN097WpN Ul+Q== X-Received: by 10.152.148.129 with SMTP id ts1mr13483997lab.19.1358658041373; Sat, 19 Jan 2013 21:00:41 -0800 (PST) Received: from dchagin.static.corbina.net (dchagin.static.corbina.ru. [78.107.232.239]) by mx.google.com with ESMTPS id f8sm3782811lbg.2.2013.01.19.21.00.39 (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 19 Jan 2013 21:00:40 -0800 (PST) Received: from dchagin.static.corbina.net (localhost [127.0.0.1]) by dchagin.static.corbina.net (8.14.6/8.14.6) with ESMTP id r0K50bWQ001591; Sun, 20 Jan 2013 09:00:37 +0400 (MSK) (envelope-from dchagin@dchagin.static.corbina.net) Received: (from dchagin@localhost) by dchagin.static.corbina.net (8.14.6/8.14.6/Submit) id r0K50blC001590; Sun, 20 Jan 2013 09:00:37 +0400 (MSK) (envelope-from dchagin) Date: Sun, 20 Jan 2013 09:00:37 +0400 From: Chagin Dmitry To: d@delphij.net Subject: Re: RFC: futimens(2) and utimensat(2) Message-ID: <20130120050037.GA1520@dchagin.static.corbina.net> References: <4F4DC876.3010809@delphij.net> <20130119145929.GA23230@dchagin.static.corbina.net> <50FB6E64.6060203@delphij.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="tThc/1wpZn/ma/RB" Content-Disposition: inline In-Reply-To: <50FB6E64.6060203@delphij.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-arch@freebsd.org, Chagin Dmitry X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jan 2013 05:00:43 -0000 --tThc/1wpZn/ma/RB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 19, 2013 at 08:11:16PM -0800, Xin Li wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 >=20 > On 1/19/13 6:59 AM, Chagin Dmitry wrote: > > On Tue, Feb 28, 2012 at 10:40:54PM -0800, Xin Li wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 > >>=20 > >> Hi, > >>=20 > >> These are required by IEEE Std 1003.1-2008. Patchset at: > >>=20 > >> http://people.freebsd.org/~delphij/for_review/utimens.diff > >>=20 > >> Cheers, - -- Xin LI > >> https://www.delphij.net/ FreeBSD - The Power to Serve! Live free > >> or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 > >> (FreeBSD) > >=20 > >=20 > > Hi Xin, Do you plan to commit this? >=20 > I'm not working on this recently, and as Jilles already pointed out, > it needs to be improved or rewritten to make sure that its standard > conformance. >=20 > Do you plan to work on this or just asking? If you have time and plan > to do it, be my guest. >=20 > Cheers, >=20 Thank you for the reply, Xin! I'll take care of it soon. --=20 Have fun! chd --tThc/1wpZn/ma/RB Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlD7efUACgkQ0t2Tb3OO/O0gHQCffK7FeD/WXlpg2VAlXiHjmldl 1tEAoInj4352FyIXJ1/tuyqGux9r4hSP =aYBE -----END PGP SIGNATURE----- --tThc/1wpZn/ma/RB-- From owner-freebsd-arch@FreeBSD.ORG Sun Jan 20 06:26:55 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5E72E5C8; Sun, 20 Jan 2013 06:26:55 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wg0-f46.google.com (mail-wg0-f46.google.com [74.125.82.46]) by mx1.freebsd.org (Postfix) with ESMTP id AA1A21ED; Sun, 20 Jan 2013 06:26:54 +0000 (UTC) Received: by mail-wg0-f46.google.com with SMTP id dr13so2981143wgb.13 for ; Sat, 19 Jan 2013 22:26:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:date:x-google-sender-auth:message-id :subject:from:to:content-type; bh=HjgSW6JXSehKtzYWuOD6wBAzREUHmgwJBgDsYj+yc90=; b=RpbySkGIh4Cb+E1o9wywNbsASZJ5puvib8iM3diJrz2PG+ncRQPceLS7OMLkKF8LyL XCPmQSBq2dbef/YorRVC0+x8XVa6BaCxDprtp40ByAq0PycoMfDeQAVett6XejIpFb+U Z9Bj7gEgsEpVzqOHY/hfJVSNKj+GydEkLMkdOSRIoOd5GndUXPdFgAj2Rnx8tlPzZu9q iu3N17PpcyDeRauoZRXB9nORQy8Crcp0aXtlsTD3dqTmb0OS3OH4AtK/upKDu9nP2Srq uwp7XGKz7YaLB6n4yuL5IhT3w8cGYaiE3Gbih/mwUdCVmjVANHna7zurXcoPqBjE4MRF FrGQ== MIME-Version: 1.0 X-Received: by 10.180.72.146 with SMTP id d18mr9993346wiv.33.1358663206802; Sat, 19 Jan 2013 22:26:46 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.217.57.9 with HTTP; Sat, 19 Jan 2013 22:26:46 -0800 (PST) Date: Sat, 19 Jan 2013 22:26:46 -0800 X-Google-Sender-Auth: Ygle3ym0DjkvbYFeLorg15ri9z8 Message-ID: Subject: [rfc] enabling MALLOC_PRODUCTION on -HEAD for now, until jemalloc has been taught to have some run time selectable debug options From: Adrian Chadd To: freebsd-arch@freebsd.org, Jason Evans Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jan 2013 06:26:55 -0000 Hi, I'd like to enable MALLOC_PRODUCTION on -HEAD. I'm currently recompiling my libc on this g4 powerbook because the -HEAD snapshots don't have it enabled by default; just to get some damned decent performance out of this thing. I'll work with Jason and others (eg Ian) who have a vested interest in trying to get it to run better out of the box, but still have the debug options available for people who wish to debug things. Thanks, Adrian From owner-freebsd-arch@FreeBSD.ORG Sun Jan 20 23:16:15 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 65606903; Sun, 20 Jan 2013 23:16:15 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 51DA7A93; Sun, 20 Jan 2013 23:16:15 +0000 (UTC) Received: from [10.0.1.102] (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218]) by elvis.mu.org (Postfix) with ESMTPSA id B24931A3C1D; Sun, 20 Jan 2013 15:16:08 -0800 (PST) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <2370711A-074E-4080-BFCF-E9C6496AD054@mu.org> X-Mailer: iPhone Mail (10A523) From: Alfred Perlstein Subject: Re: [rfc] enabling MALLOC_PRODUCTION on -HEAD for now, until jemalloc has been taught to have some run time selectable debug options Date: Sun, 20 Jan 2013 15:16:06 -0800 To: Adrian Chadd Cc: Jason Evans , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jan 2013 23:16:15 -0000 For the time being how about setting it for these smaller platforms? PowerP= C and maybe MIPS? Sent from my iPhone On Jan 19, 2013, at 10:26 PM, Adrian Chadd wrote: > Hi, >=20 > I'd like to enable MALLOC_PRODUCTION on -HEAD. >=20 > I'm currently recompiling my libc on this g4 powerbook because the > -HEAD snapshots don't have it enabled by default; just to get some > damned decent performance out of this thing. >=20 > I'll work with Jason and others (eg Ian) who have a vested interest in > trying to get it to run better out of the box, but still have the > debug options available for people who wish to debug things. >=20 > Thanks, >=20 >=20 >=20 > Adrian > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >=20 From owner-freebsd-arch@FreeBSD.ORG Sun Jan 20 23:39:09 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 7FF86C34; Sun, 20 Jan 2013 23:39:09 +0000 (UTC) (envelope-from nwhitehorn@freebsd.org) Received: from agogare.doit.wisc.edu (agogare.doit.wisc.edu [144.92.197.211]) by mx1.freebsd.org (Postfix) with ESMTP id 376B3B4F; Sun, 20 Jan 2013 23:39:09 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from wanderer.tachypleus.net (adsl-76-208-68-53.dsl.mdsnwi.sbcglobal.net [76.208.68.53]) by smtpauth2.wiscmail.wisc.edu (Sun Java(tm) System Messaging Server 7u2-7.05 32bit (built Jul 30 2009)) with ESMTPSA id <0MGY001JK7150E30@smtpauth2.wiscmail.wisc.edu>; Sun, 20 Jan 2013 17:39:08 -0600 (CST) X-Spam-PmxInfo: Server=avs-2, Version=5.6.1.2065439, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2013.1.20.232716, SenderIP=76.208.68.53 X-Spam-Report: AuthenticatedSender=yes, SenderIP=76.208.68.53 X-Wisc-Sender: whitehorn@wisc.edu Message-id: <50FC8019.40706@freebsd.org> Date: Sun, 20 Jan 2013 17:39:05 -0600 From: Nathan Whitehorn User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130112 Thunderbird/17.0.2 To: Alfred Perlstein Subject: Re: [rfc] enabling MALLOC_PRODUCTION on -HEAD for now, until jemalloc has been taught to have some run time selectable debug options References: <2370711A-074E-4080-BFCF-E9C6496AD054@mu.org> In-reply-to: <2370711A-074E-4080-BFCF-E9C6496AD054@mu.org> Cc: Adrian Chadd , Jason Evans , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jan 2013 23:39:09 -0000 It's fatal for being able to do *anything* on all machines but the very fanciest and newest. I forgot it on my Core 2 Duo laptop and regretted it thoroughly -- especially since undoing the mistake involved rebuilding world on a now *very* slow machine. I'm also not sure we lose anything by turning it on by default as it currently is set. Anyone actually running -CURRENT day to day has MALLOC_PRODUCTION enabled so the tests are never seen anyway. -Nathan On 01/20/13 17:16, Alfred Perlstein wrote: > For the time being how about setting it for these smaller platforms? PowerPC and maybe MIPS? > > Sent from my iPhone > > On Jan 19, 2013, at 10:26 PM, Adrian Chadd wrote: > >> Hi, >> >> I'd like to enable MALLOC_PRODUCTION on -HEAD. >> >> I'm currently recompiling my libc on this g4 powerbook because the >> -HEAD snapshots don't have it enabled by default; just to get some >> damned decent performance out of this thing. >> >> I'll work with Jason and others (eg Ian) who have a vested interest in >> trying to get it to run better out of the box, but still have the >> debug options available for people who wish to debug things. >> >> Thanks, >> >> >> >> Adrian >> _______________________________________________ >> freebsd-arch@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >> To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >> > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Mon Jan 21 03:26:43 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A2D74265; Mon, 21 Jan 2013 03:26:43 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 784526A0; Mon, 21 Jan 2013 03:26:43 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id r0L3Qggw041969; Sun, 20 Jan 2013 20:26:42 -0700 (MST) (envelope-from ian@FreeBSD.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r0L3QJIC010991; Sun, 20 Jan 2013 20:26:19 -0700 (MST) (envelope-from ian@FreeBSD.org) Subject: Re: [rfc] enabling MALLOC_PRODUCTION on -HEAD for now, until jemalloc has been taught to have some run time selectable debug options From: Ian Lepore To: Adrian Chadd In-Reply-To: References: Content-Type: multipart/mixed; boundary="=-pHExhQ4eXymyCAIGbJ0z" Date: Sun, 20 Jan 2013 20:26:19 -0700 Message-ID: <1358738779.32417.380.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Cc: Jason Evans , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2013 03:26:43 -0000 --=-pHExhQ4eXymyCAIGbJ0z Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Sat, 2013-01-19 at 22:26 -0800, Adrian Chadd wrote: > Hi, > > I'd like to enable MALLOC_PRODUCTION on -HEAD. > > I'm currently recompiling my libc on this g4 powerbook because the > -HEAD snapshots don't have it enabled by default; just to get some > damned decent performance out of this thing. > > I'll work with Jason and others (eg Ian) who have a vested interest in > trying to get it to run better out of the box, but still have the > debug options available for people who wish to debug things. > I've been investigating this today and have some information. With MALLOC_PRODUCTION defined there is no problem, even on small embedded systems. Without MALLOC_PRODUCTION we've basically got two problems: * Every program has a minimum resident size of about 8MiB, and that's fatal on a small-memory embedded system. * Performance is bad. This is at least in part due to the expense of faulting in 8MiB of zeroed pages, and that's especially noticible in utilities that should be small and fast. There could be other causes as well. I think I've tracked the cause of the 8MiB resident size to a particular sanity check, which validates whether memory that was supposed to have been zeroed actually was. I think this check makes sense in some cases, and not in others. It almost certainly doesn't make sense if the memory was freshly obtained from mmap(). I want to talk to Jason about a proper robust fix, but to help learn more about the performance problem, I'm attaching a little test patch that disables the suspect validity check. It would be good if a few folks running -current could apply this and build without MALLOC_PRODUCTION defined, and see if the system feels more usable than it does without the patch. It's likely to make the most difference on a slower or older system. It's possible that this patch helps with the memory usage, but doesn't help enough with performance. I'm not in a good position to do real-world performance testing myself right now. In terms of non-real-world testing, I was using a trivial little app that was basically: int main(void) {malloc(64); return 0;} and a little shell script to time running 100 iterations of that in a loop. Without the patch it took 24 seconds, with the patch 2 seconds, on a medium-wimpy embeded arm system. It's probably too much to hope that a 12:1 improvement will scale up to non-trivial apps. -- Ian --=-pHExhQ4eXymyCAIGbJ0z Content-Disposition: inline; filename="jemalloc_test.diff" Content-Type: text/x-patch; name="jemalloc_test.diff"; charset="us-ascii" Content-Transfer-Encoding: 7bit Index: contrib/jemalloc/src/chunk.c =================================================================== --- contrib/jemalloc/src/chunk.c (revision 245695) +++ contrib/jemalloc/src/chunk.c (working copy) @@ -195,12 +203,7 @@ prof_gdump(); } if (config_debug && *zero && ret != NULL) { - size_t i; - size_t *p = (size_t *)(uintptr_t)ret; - VALGRIND_MAKE_MEM_DEFINED(ret, size); - for (i = 0; i < size / sizeof(size_t); i++) - assert(p[i] == 0); } assert(CHUNK_ADDR2BASE(ret) == ret); return (ret); --=-pHExhQ4eXymyCAIGbJ0z-- From owner-freebsd-arch@FreeBSD.ORG Mon Jan 21 04:38:11 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 613ADF78 for ; Mon, 21 Jan 2013 04:38:11 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 1D71E92B for ; Mon, 21 Jan 2013 04:38:07 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id r0L4c6eQ042780 for ; Sun, 20 Jan 2013 21:38:07 -0700 (MST) (envelope-from ian@FreeBSD.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r0L4bisF011039; Sun, 20 Jan 2013 21:37:44 -0700 (MST) (envelope-from ian@FreeBSD.org) Subject: Re: RFC: enhanced watchdog. From: Ian Lepore To: Alfred Perlstein In-Reply-To: <50FA3D36.4080709@mu.org> References: <201301190604.r0J64RbW009298@svn.freebsd.org> <50FA3D36.4080709@mu.org> Content-Type: text/plain; charset="us-ascii" Date: Sun, 20 Jan 2013 21:37:44 -0700 Message-ID: <1358743064.32417.409.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: "arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2013 04:38:11 -0000 On Fri, 2013-01-18 at 22:29 -0800, Alfred Perlstein wrote: > We at iX are trying to enhance the watchdog and we think some of the > changes may benefit the community as a whole. > > Basically we want to make it easy for developers to prototype watchdog > scripts in a "test-only" mode that basically logs if the watchdog had > failed. > > I have most of the code done, but could really use help on two things: > > 1) review > 2) suggestion for inserting the warning messages from the userland > watchdogd into the kernel message buffer. > 3) suggestion for logging/warning of pending death. > > In detail: > 1) The reason for review should be obvious, we want to make sure that > this works for everyone. > 2) The reason for inserting messages into the kernel log is because that > is the easiest place for us to recover the diagnostics when we do have a > crash due to watchdog. Maybe there is a smarter thing to do? I've recently wished for a way that a sufficiently-credentialed userland process could, in effect, kernel-printf. I've been burned a number of times by init(8) failing to start up for various reasons such as no /dev, and it has no way to say what's wrong. It's surprisingly hard to figure out what the problem is. For your need, a possiblity I guess would be to have the watchdog device do it for you, since you're already talking to it. Who knows, maybe some special watchdog hardware would be able to do something useful with a short message. I've worked with hardware that has a few registers designed to survive a reboot, for communicating with your reincarnated self; nothing big enough for arbitrary strings yet, but hardware just keeps getting cooler all the time. > 3) What is a good way to warn of impeding death? I was thinking of just > another thread in the process that would be signalled before the > watchdog script was run and would log when the timer is about to expire > or based on a configurable threshold. > SIGALRM that fires shortly before death? > > Finally, there is some thought about adding a kernel daemon to the > watchdog facility that would allow us to strobe watchdogs with low max > values while our userland watchdog was polling the system. > > Why??? Well because the ICH driver has a max timeout of ~2 minutes. We > really want to be able to leverage this watchdog, but also go higher > than this. The way to do this is to drive the system almost like a step > up electrical relay. > I very much like this. A new ARM SoC I'm about to start working with has a max 16 second watchdog, and I'm afraid things like firmware updaters might lock out userland for longer than that on such a wimpy chip. > [... code ...] I skimmed through the code, but it's been a long day of reading code for me, so I'm not gonna pretend it was a thorough review. The main thing that popped out at me was 'carp'. Shouldn't a watchdog bark? :) I'm also curious why you chose CLOCK_UPTIME_FAST, which I'm not familiar with (gonna be reading a manpage in a minute). Not knowing about some of the newer choices, I probably would've used CLOCK_MONOTONIC. -- Ian From owner-freebsd-arch@FreeBSD.ORG Mon Jan 21 07:22:38 2013 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id CCA3FC44; Mon, 21 Jan 2013 07:22:38 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id AB4F5F15; Mon, 21 Jan 2013 07:22:38 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218]) by elvis.mu.org (Postfix) with ESMTPSA id E32591A3C1D; Sun, 20 Jan 2013 23:22:37 -0800 (PST) Message-ID: <50FCECBD.9090002@mu.org> Date: Sun, 20 Jan 2013 23:22:37 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Ian Lepore Subject: Re: RFC: enhanced watchdog. References: <201301190604.r0J64RbW009298@svn.freebsd.org> <50FA3D36.4080709@mu.org> <1358743064.32417.409.camel@revolution.hippie.lan> In-Reply-To: <1358743064.32417.409.camel@revolution.hippie.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2013 07:22:38 -0000 On 1/20/13 8:37 PM, Ian Lepore wrote: > On Fri, 2013-01-18 at 22:29 -0800, Alfred Perlstein wrote: >> We at iX are trying to enhance the watchdog and we think some of the >> changes may benefit the community as a whole. >> >> Basically we want to make it easy for developers to prototype watchdog >> scripts in a "test-only" mode that basically logs if the watchdog had >> failed. >> >> I have most of the code done, but could really use help on two things: >> >> 1) review >> 2) suggestion for inserting the warning messages from the userland >> watchdogd into the kernel message buffer. >> 3) suggestion for logging/warning of pending death. >> >> In detail: >> 1) The reason for review should be obvious, we want to make sure that >> this works for everyone. >> 2) The reason for inserting messages into the kernel log is because that >> is the easiest place for us to recover the diagnostics when we do have a >> crash due to watchdog. Maybe there is a smarter thing to do? > I've recently wished for a way that a sufficiently-credentialed userland > process could, in effect, kernel-printf. I've been burned a number of > times by init(8) failing to start up for various reasons such as > no /dev, and it has no way to say what's wrong. It's surprisingly hard > to figure out what the problem is. > > For your need, a possiblity I guess would be to have the watchdog device > do it for you, since you're already talking to it. Who knows, maybe > some special watchdog hardware would be able to do something useful with > a short message. I've worked with hardware that has a few registers > designed to survive a reboot, for communicating with your reincarnated > self; nothing big enough for arbitrary strings yet, but hardware just > keeps getting cooler all the time. I'm almost wondering if there's some kind of /dev/klog we should/could have? > >> 3) What is a good way to warn of impeding death? I was thinking of just >> another thread in the process that would be signalled before the >> watchdog script was run and would log when the timer is about to expire >> or based on a configurable threshold. >> > SIGALRM that fires shortly before death? That sounds great. I'll look into that. > >> Finally, there is some thought about adding a kernel daemon to the >> watchdog facility that would allow us to strobe watchdogs with low max >> values while our userland watchdog was polling the system. >> >> Why??? Well because the ICH driver has a max timeout of ~2 minutes. We >> really want to be able to leverage this watchdog, but also go higher >> than this. The way to do this is to drive the system almost like a step >> up electrical relay. >> > I very much like this. A new ARM SoC I'm about to start working with > has a max 16 second watchdog, and I'm afraid things like firmware > updaters might lock out userland for longer than that on such a wimpy > chip. > >> [... code ...] > I skimmed through the code, but it's been a long day of reading code for > me, so I'm not gonna pretend it was a thorough review. The main thing > that popped out at me was 'carp'. Shouldn't a watchdog bark? :) ha! > > I'm also curious why you chose CLOCK_UPTIME_FAST, which I'm not familiar > with (gonna be reading a manpage in a minute). Not knowing about some > of the newer choices, I probably would've used CLOCK_MONOTONIC. I unfortunately am a generalist and my clock-fu is weak. I can look into switching to that. What would be the difference between the two in general? -Alfred From owner-freebsd-arch@FreeBSD.ORG Mon Jan 21 09:07:30 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DC5716D4; Mon, 21 Jan 2013 09:07:30 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) by mx1.freebsd.org (Postfix) with ESMTP id 4BDD3958; Mon, 21 Jan 2013 09:07:29 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.6/8.14.6) with ESMTP id r0L97L2U032749; Mon, 21 Jan 2013 13:07:21 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.6/8.14.6/Submit) id r0L97LxR032748; Mon, 21 Jan 2013 13:07:21 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Mon, 21 Jan 2013 13:07:21 +0400 From: Gleb Smirnoff To: Nathan Whitehorn Subject: Re: [rfc] enabling MALLOC_PRODUCTION on -HEAD for now, until jemalloc has been taught to have some run time selectable debug options Message-ID: <20130121090721.GO14114@FreeBSD.org> References: <2370711A-074E-4080-BFCF-E9C6496AD054@mu.org> <50FC8019.40706@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <50FC8019.40706@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Adrian Chadd , Alfred Perlstein , Jason Evans , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2013 09:07:30 -0000 On Sun, Jan 20, 2013 at 05:39:05PM -0600, Nathan Whitehorn wrote: N> I'm also not sure we lose anything by turning it on by default as it N> currently is set. Anyone actually running -CURRENT day to day has N> MALLOC_PRODUCTION enabled so the tests are never seen anyway. Definitely not true. -- Totus tuus, Glebius. From owner-freebsd-arch@FreeBSD.ORG Mon Jan 21 09:54:59 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AA844622; Mon, 21 Jan 2013 09:54:59 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id 0743CB81; Mon, 21 Jan 2013 09:54:58 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.5/8.14.5/ALCHEMY.FRANKEN.DE) with ESMTP id r0L9svRW099017; Mon, 21 Jan 2013 10:54:57 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.5/8.14.5/Submit) id r0L9sv57099016; Mon, 21 Jan 2013 10:54:57 +0100 (CET) (envelope-from marius) Date: Mon, 21 Jan 2013 10:54:57 +0100 From: Marius Strobl To: Alexander Motin Subject: Re: [RFC/RFT] calloutng Message-ID: <20130121095457.GL85306@alchemy.franken.de> References: <50CCAB99.4040308@FreeBSD.org> <50CE5B54.3050905@FreeBSD.org> <50D03173.9080904@FreeBSD.org> <20121225232126.GA47692@alchemy.franken.de> <50DB4EFE.2020600@FreeBSD.org> <20130106152313.GD26039@alchemy.franken.de> <50EBF921.2000304@FreeBSD.org> <20130113180940.GM26039@alchemy.franken.de> <50F30CAB.3000001@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50F30CAB.3000001@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Davide Italiano , FreeBSD Current , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2013 09:54:59 -0000 On Sun, Jan 13, 2013 at 09:36:11PM +0200, Alexander Motin wrote: > On 13.01.2013 20:09, Marius Strobl wrote: > > On Tue, Jan 08, 2013 at 12:46:57PM +0200, Alexander Motin wrote: > >> On 06.01.2013 17:23, Marius Strobl wrote: > >>> I'm not really sure what to do about that. Earlier you already said > >>> that sched_bind(9) also isn't an option in case if td_critnest > 1. > >>> To be honest, I don't really unerstand why using a spin lock in the > >>> timecounter path makes sparc64 the only problematic architecture > >>> for your changes. The x86 i8254_get_timecount() also uses a spin lock > >>> so it should be in the same boat. > >> > >> The problem is not in using spinlock, but in waiting for other CPU while > >> spinlock is held. Other CPU may also hold spinlock and wait for > >> something, causing deadlock. i8254 code uses spinlock just to atomically > >> access hardware registers, so it causes no problems. > > > > Okay, but wouldn't that be a general problem then? Pretty much > > anything triggering an IPI holds smp_ipi_mtx while doing so and > > the lower level IPI stuff waits for other CPU(s), including on > > x86. > > The problem is general. But now it works because single smp_ipi_mtx is > used in all cases where IPI result is waited. As soon as spinning > happens with interrupts still enabled, there is no deadlocks. But > problem reappears if any different lock is used, or locks are nested. I'm having a hard time getting an alternate time counter device to work. The crystal required for the counters in the south bridge just doesn't seem to be mounted any where near it (I've not looked at the bottom of the PCB though). While the time counter part of the on- board bge(4) driven chips basically work, they don't seem to like concurrent accesses caused by the rest of bge(4). I.e. although the counter is just read, sooner or later this causes a fatal bus error. I haven't tried serializing accesses to the chip, but getting to such a complexity for just reading a non-indexed register at least doesn't feel good ... However, AFAICT the scenario you describe can't happen. On sparc64, spinlock_enter() only raises the processor interrupt level, which doesn't block the direct cross traps I've implemented remote reading of (S)TICK as (which also means that the actions such traps may perform are very limitted and must occur in interrupt context, but which are sufficient for this purpose and in turn makes them very fast). I.e. although the AP holds smp_ipi_mtx or any amount of nested spin locks, this will not deadlock in case the BSP also holds any spin lock when reading (S)TICK from it. Marius From owner-freebsd-arch@FreeBSD.ORG Mon Jan 21 19:00:02 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 11290547; Mon, 21 Jan 2013 19:00:02 +0000 (UTC) (envelope-from jasone@freebsd.org) Received: from canonware.com (canonware.com [204.109.63.53]) by mx1.freebsd.org (Postfix) with ESMTP id CC64980C; Mon, 21 Jan 2013 19:00:01 +0000 (UTC) Received: from [192.168.168.12] (70-91-206-178-BusName-SFBA.hfc.comcastbusiness.net [70.91.206.178]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by canonware.com (Postfix) with ESMTPSA id 0FBC52842D; Mon, 21 Jan 2013 11:00:01 -0800 (PST) Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: [rfc] enabling MALLOC_PRODUCTION on -HEAD for now, until jemalloc has been taught to have some run time selectable debug options From: Jason Evans In-Reply-To: <1358738779.32417.380.camel@revolution.hippie.lan> Date: Mon, 21 Jan 2013 11:00:00 -0800 Message-Id: <96396542-ABE9-44DC-9F3C-9098434E7EE5@freebsd.org> References: <1358738779.32417.380.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1499) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Adrian Chadd , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2013 19:00:02 -0000 On Jan 20, 2013, at 7:26 PM, Ian Lepore wrote: > On Sat, 2013-01-19 at 22:26 -0800, Adrian Chadd wrote: >> I'd like to enable MALLOC_PRODUCTION on -HEAD. >>=20 >> I'm currently recompiling my libc on this g4 powerbook because the >> -HEAD snapshots don't have it enabled by default; just to get some >> damned decent performance out of this thing. >>=20 >> I'll work with Jason and others (eg Ian) who have a vested interest = in >> trying to get it to run better out of the box, but still have the >> debug options available for people who wish to debug things. >=20 > I've been investigating this today and have some information. >=20 > With MALLOC_PRODUCTION defined there is no problem, even on small > embedded systems. Without MALLOC_PRODUCTION we've basically got two > problems: >=20 > * Every program has a minimum resident size of about 8MiB, and > that's fatal on a small-memory embedded system. > * Performance is bad. This is at least in part due to the = expense > of faulting in 8MiB of zeroed pages, and that's especially > noticible in utilities that should be small and fast. There > could be other causes as well. >=20 > I think I've tracked the cause of the 8MiB resident size to a = particular > sanity check, which validates whether memory that was supposed to have > been zeroed actually was. I think this check makes sense in some = cases, > and not in others. It almost certainly doesn't make sense if the = memory > was freshly obtained from mmap(). Now I remember planning to remove the zeroed memory validation because = of its high cost, but deciding otherwise at the last moment. Literally = the day before I released jemalloc 3.0.0, the validation code caught = this (in the context of FreeBSD's libc): Fix large calloc() zeroing bugs. = http://www.canonware.com/cgi-bin/gitweb.cgi?p=3Djemalloc.git;a=3Dcommitdif= f;h=3Dd8ceef6c5558fdab8f9448376ae065a9e5ffcbdd Since 3.0.0 though there has been some refactoring that makes it = feasible (and clean) to move the validation code from chunk_alloc() to = chunk_recycle(). Ian sent me such a patch (better than the one I would = have come up with myself) that I will test and integrate upstream. In = the meanwhile I think we should try Ian's patch before resorting to = MALLOC_PRODUCTION. Jason= From owner-freebsd-arch@FreeBSD.ORG Thu Jan 24 18:40:22 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E769C385; Thu, 24 Jan 2013 18:40:22 +0000 (UTC) (envelope-from jasone@freebsd.org) Received: from canonware.com (canonware.com [204.109.63.53]) by mx1.freebsd.org (Postfix) with ESMTP id CCA89D93; Thu, 24 Jan 2013 18:40:22 +0000 (UTC) Received: from jemac.thefacebook.com (unknown [173.252.71.4]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by canonware.com (Postfix) with ESMTPSA id 78D0428417; Thu, 24 Jan 2013 10:40:16 -0800 (PST) Subject: Re: [rfc] enabling MALLOC_PRODUCTION on -HEAD for now, until jemalloc has been taught to have some run time selectable debug options Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=iso-8859-1 From: Jason Evans In-Reply-To: Date: Thu, 24 Jan 2013 10:40:15 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Adrian Chadd X-Mailer: Apple Mail (2.1283) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2013 18:40:23 -0000 On Jan 19, 2013, at 10:26 PM, Adrian Chadd wrote: > I'd like to enable MALLOC_PRODUCTION on -HEAD. >=20 > I'm currently recompiling my libc on this g4 powerbook because the > -HEAD snapshots don't have it enabled by default; just to get some > damned decent performance out of this thing. >=20 > I'll work with Jason and others (eg Ian) who have a vested interest in > trying to get it to run better out of the box, but still have the > debug options available for people who wish to debug things. I imported jemalloc 3.3.0 last night, which I think will restore = non-MALLOC_PRODUCTION performance to roughly what it has been for years. = The key change was a patch by Ian Lepore: = http://www.canonware.com/cgi-bin/gitweb.cgi?p=3Djemalloc.git;a=3Dcommitdif= f;h=3D14a2c6a698a207ac3f3825443cf3441c8842e990 In practice, the zeroed memory validation should no longer execute at = all on FreeBSD due to madvise(...MADV_FREE) not guaranteeing zeroed = memory. I don't know how significant the impact will be on constrained/older = systems, but for the many buildworld repetitions I did during testing it = improved throughput by >1.5X. With this change in place, I'm not keen = on enabling MALLOC_PRODUCTION by default, because the assertions are = useful both for detecting bugs in jemalloc (heaven forbid), and for = indicating that crashes within jemalloc are application errors. The = only other significant source of overhead I'm aware of is junk filling, = which we've had enabled by default on -current for ages, probably since = the introduction of phkmalloc. If you don't like it: ln -s 'junk:false' /etc/malloc.conf If a substantial performance difference remains for you = (MALLOC_PRODUCTION vs non-MALLOC_PRODUCTION with junk:false set), please = let me know. Thanks, Jason= From owner-freebsd-arch@FreeBSD.ORG Thu Jan 24 20:30:43 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 93E81418; Thu, 24 Jan 2013 20:30:43 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wi0-f172.google.com (mail-wi0-f172.google.com [209.85.212.172]) by mx1.freebsd.org (Postfix) with ESMTP id EDB732F6; Thu, 24 Jan 2013 20:30:42 +0000 (UTC) Received: by mail-wi0-f172.google.com with SMTP id o1so827747wic.11 for ; Thu, 24 Jan 2013 12:30:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=oNEjBDexpeKomx2Ocx3EubxQ2khCB115kqebUHv2xk4=; b=bJLGaU3TOs691EDbQwo/Fg9UpveGQgQyca1av2tlgEKRhmYfL4YVhLjZqeKxvaK3b3 +c7ST2N+PCVbEa8JiNxthSlXwWl58TZDe5kOu+volQskDFrAku0REEjnA3BKkCBLxMju 0Fwu7FY2DyfUDpvr7vuzR2DCx2qm+8E7jUYRwjFyu3CFDY3PuyPXszpkIhKSsXrv89Tc 7q9hsH1UJPUPGhuJrIXw4nw22jJGdkrgIdZEnsNi8KPmFLXrpySrNaoPuQJjLV3+odMY 5DGzUSz7qhswmefPA2k4s085hsZKB8VSRqn8TiKM/u8YlbP/AIyNPqtAwJ74zcLSQlaQ HoEw== MIME-Version: 1.0 X-Received: by 10.194.172.197 with SMTP id be5mr5379504wjc.20.1359059436389; Thu, 24 Jan 2013 12:30:36 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.216.67.6 with HTTP; Thu, 24 Jan 2013 12:30:36 -0800 (PST) In-Reply-To: References: Date: Thu, 24 Jan 2013 12:30:36 -0800 X-Google-Sender-Auth: 4vioMybD9jMiOpATRRb9ogltFKY Message-ID: Subject: Re: [rfc] enabling MALLOC_PRODUCTION on -HEAD for now, until jemalloc has been taught to have some run time selectable debug options From: Adrian Chadd To: Jason Evans Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2013 20:30:43 -0000 On 24 January 2013 10:40, Jason Evans wrote: > I imported jemalloc 3.3.0 last night, which I think will restore non-MALL= OC_PRODUCTION performance to roughly what it has been for years. The key c= hange was a patch by Ian Lepore: > > http://www.canonware.com/cgi-bin/gitweb.cgi?p=3Djemalloc.git;a=3D= commitdiff;h=3D14a2c6a698a207ac3f3825443cf3441c8842e990 > > In practice, the zeroed memory validation should no longer execute at all= on FreeBSD due to madvise(...MADV_FREE) not guaranteeing zeroed memory. > > I don't know how significant the impact will be on constrained/older syst= ems, but for the many buildworld repetitions I did during testing it improv= ed throughput by >1.5X. With this change in place, I'm not keen on enablin= g MALLOC_PRODUCTION by default, because the assertions are useful both for = detecting bugs in jemalloc (heaven forbid), and for indicating that crashes= within jemalloc are application errors. The only other significant source= of overhead I'm aware of is junk filling, which we've had enabled by defau= lt on -current for ages, probably since the introduction of phkmalloc. If = you don't like it: > > ln -s 'junk:false' /etc/malloc.conf > > If a substantial performance difference remains for you (MALLOC_PRODUCTIO= N vs non-MALLOC_PRODUCTION with junk:false set), please let me know. Sweet, thanks. I'll do up a new -HEAD MIPS, ARM and PPC test builds tonight and let you know how it all goes. I'd love to keep MALLOC_PRODUCTION off on -HEAD by default. :-) Thank you to both you and Ian for spearheading this change to your malloc library. :) Adrian