Date: Fri, 16 Jan 2004 05:14:21 -0800 From: Mike Harding <mvh@ix.netcom.com> To: Alexander Leidinger <Alexander@Leidinger.net> Cc: ports@freebsd.org Subject: Re: perl 5.8 and malloc (and mod_perl) Message-ID: <1074258860.684.45.camel@netcom1.netcom.com> In-Reply-To: <20040116122858.278c2527@Magellan.Leidinger.net> References: <20040115233128.BE39954E5@netcom1.netcom.com> <20040116122858.278c2527@Magellan.Leidinger.net>
next in thread | previous in thread | raw e-mail | index | archive | help
(much of the following is, of course, FreeBSD specific, if you found this via Google) Note that perl 5.8+ has "safe signals" which defer the signal processing until the next perl opcode - apparently entering the signal information into a queue. This should allow non-perl malloc to be used. See PERLIPC(1) for 5.8.2 - it discusses this in detail. Note that Perl signals were never really -ever- safe before, as far as I can tell, unless you went to great lengths to make sure that no non-reentrant code was called - and the Perl system code is/was apparently not very reentrant. So, I think that I am (much) better off using system malloc with mod_perl, at least if you do log rotation with grace restarts. If you don't use mod_perl then either should work, and the perl_malloc should be faster, at least in some edge cases, at the expense of some memory. There was some discussion of using system malloc with Perl degenerating into n^2 performance if buffers were extended 1 unit at a time, as unzip apparently does, as perl would call realloc n times - the Perl malloc use(s/d) exponential reallocation to avoid this. This could be fixed in the Perl system source code so that both systems work the same, but the edge goes to Perl's malloc for speed for now. I'm just trying to give mod_perl users this info because, at least for the DSO mod_perl case, using the system malloc appears to fix a massive memory leak during an apache graceful restart, which is a common way to rotate Apache log files. For other uses, Perl's malloc is probably superior. It may also be possible to stop the leak by building Apache or Perl some other way so that Perl's malloc can be used, but I'm not interested in looking at this unless some problems arise from using the system malloc. I don't think that the speed problem mentioned above will affect systems that I work on - the systems, at least initially, appear as fast or faster than before I used the system malloc with Perl/mod_perl. Note that this should not affect people who don't use graceful restarts for log rotation, although it does appear that the httpd process is much smaller (1/2 or 2/3 of the size) if the system malloc is used. This may be a genuine leak, or just how Perl uses memory or appears to use memory, but I am happy to see Mason running over mod_perl with a 14 meg httpd. I found this interesting tidbit here: http://www.nntp.perl.org/group/perl.perl5.summary/61 "Using perl's malloc() on FreeBSD produce spurious failures in the threading stress tests. Consequently, Jarkko reverted perl to use FreeBSD's malloc(), in spite of its slowness (see Dan Kogai's analysis in last week summary.)" On Fri, 2004-01-16 at 03:28, Alexander Leidinger wrote: > On Thu, 15 Jan 2004 15:31:28 -0800 (PST) > Mike Harding <mvh@ix.netcom.com> wrote: > > > > > (This is a copy of a mail already sent to the perl5.8 port maintainer, > > I wanted to make sure that this info is available to those who run > > into the same issue) > > > > I have a few client sites where recently (since a perl upgrade) > > Mason/mod_perl based httpd processes had 'swollen' to huge sizes - > > 300+ meg. After a lot of coffee and googling and such, it looks like > > the culprit is somehow related to the use of perl's malloc. When > > apache gets a graceful restart signal (for rotating logs) the root > > apache process (and all it's children) grow by 6 megs or so. I'm > > rotating the logs every day at midnight so it took a while for me to > > notice. > > I don't know if the situation changed with perl 5.8, but with 5.6 you > have to compile with perls malloc if your application makes use of > signals, else you will see heisenbugs. It isn't safe to call malloc in a > signal handler -- see sigaction(2) -- and a perl program which makes > use of signals may just do this. > > Bye, > Alexander.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1074258860.684.45.camel>