From owner-freebsd-current@FreeBSD.ORG Thu Jan 3 06:00:22 2008 Return-Path: Delivered-To: current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BCFE116A418; Thu, 3 Jan 2008 06:00:22 +0000 (UTC) (envelope-from marcus@FreeBSD.org) Received: from creme-brulee.marcuscom.com (marcuscom-pt.tunnel.tserv1.fmt.ipv6.he.net [IPv6:2001:470:1f00:ffff::1279]) by mx1.freebsd.org (Postfix) with ESMTP id 8C0AE13C469; Thu, 3 Jan 2008 06:00:22 +0000 (UTC) (envelope-from marcus@FreeBSD.org) Received: from [IPv6:2001:470:1f00:2464::4] (shumai.marcuscom.com [IPv6:2001:470:1f00:2464::4]) by creme-brulee.marcuscom.com (8.14.2/8.14.2) with ESMTP id m0361FOU093507; Thu, 3 Jan 2008 01:01:15 -0500 (EST) (envelope-from marcus@FreeBSD.org) From: Joe Marcus Clarke To: Jason Evans In-Reply-To: <477C47BC.1020101@freebsd.org> References: <1199314166.9913.63.camel@shumai.marcuscom.com> <477C47BC.1020101@freebsd.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-fXfxdS28cvSaXkQrWs6n" Organization: FreeBSD, Inc. Date: Thu, 03 Jan 2008 01:00:28 -0500 Message-Id: <1199340028.64371.9.camel@shumai.marcuscom.com> Mime-Version: 1.0 X-Mailer: Evolution 2.12.2 FreeBSD GNOME Team Port X-Spam-Status: No, score=-1.2 required=5.0 tests=BAYES_00,MIME_QP_LONG_LINE, NO_RELAYS autolearn=no version=3.2.3 X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on creme-brulee.marcuscom.com Cc: current Subject: Re: Memory problem with latest malloc.c X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jan 2008 06:00:22 -0000 --=-fXfxdS28cvSaXkQrWs6n Content-Type: text/plain Content-Transfer-Encoding: quoted-printable On Wed, 2008-01-02 at 18:26 -0800, Jason Evans wrote: > Joe Marcus Clarke wrote: > > GNOME applications use a Python driven XML parser to generate help > > document translations. The engine takes the English XML document and > > applies translations to it. The tool, xml2po, is installed as part of > > textproc/gnome-doc-utils. I'm currently working on porting GNOME 2.21, > > and one of the Evolution help doc changes triggered a memory problem on > > my test machine. Basically, with up to and including rev 1.154 of > > malloc.c, I am able to generate the help file with no errors. With all > > later revs including 1.160, the Python process balloons up to about 512 > > MB of memory, then dies. > >=20 > > The only malloc config I've done is symlink Aj to /etc/malloc.conf. I'= d > > be happy to provide the file an exact command used, but it might be > > easier to let me know if there's any debugging I could provide that > > would help here. >=20 > Revision 1.160 of malloc.c uses sbrk instead of mmap by default. The=20 > intent is to make data segment resource limits useful. So, there are=20 > two possible explanations I can think of for your problems. The first=20 > possibility is that there is still a bug in malloc that you're hitting.=20 > However, the other possibility is that the python program you're=20 > running really needs more than 512 MB of memory, and you're hitting the=20 > resource limit. >=20 > It would be really helpful to me if you run your program with=20 > MALLOC_OPTIONS=3DdM and monitor memory usage. These flags cause mmap to=20 > be used instead of sbrk, and we can find out from that how much memory=20 > you really need. If peak memory usage is substantially different when=20 > using mmap versus sbrk, there's probably a malloc bug. Memory climbed up to 976 MB SZ, 974 MB RSS MB with dM -> /etc/malloc.conf. The file was eventually generated without error. Again, with Aj -> /etc/malloc.conf, the python2.5 process operating on the same file planed out at 504 MB SZ, 501 MB RSS. >=20 > This latest round of malloc changes hasn't been much fun. Thanks for=20 > your help and patience. No problem. Thanks for getting back to me. Joe --=20 Joe Marcus Clarke FreeBSD GNOME Team :: gnome@FreeBSD.org FreeNode / #freebsd-gnome http://www.FreeBSD.org/gnome --=-fXfxdS28cvSaXkQrWs6n Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQBHfHn4b2iPiv4Uz4cRAn8lAJ0dcD9m25vEIarrzycSWt6IAG3pqgCePnd+ aO5kMhL6wqPX8J46sKeb/gg= =Jg5E -----END PGP SIGNATURE----- --=-fXfxdS28cvSaXkQrWs6n--