From owner-freebsd-stable Fri May 1 21:44:06 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id VAA05212 for freebsd-stable-outgoing; Fri, 1 May 1998 21:44:06 -0700 (PDT) (envelope-from owner-freebsd-stable@FreeBSD.ORG) Received: from chardos.connix.com (x7qxmWFxWh0uWdDKpj5ytweKUPNSiKai@chardos.connix.com [205.246.97.17]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id VAA05052 for ; Fri, 1 May 1998 21:43:56 -0700 (PDT) (envelope-from saken@chardos.connix.com) Received: (from saken@localhost) by chardos.connix.com (8.8.8/8.8.8) id AAA20534; Sat, 2 May 1998 00:43:44 -0400 (EDT) (envelope-from saken) Message-ID: <19980502004341.14009@rmta.ml.org> Date: Sat, 2 May 1998 00:43:41 -0400 From: "Scott A. Kenney" To: Berend de Boer Cc: "'FreeBSD stable'" Subject: Re: swap_pager: out of swap space, only power off did help References: <01BD754F.A0AB9EF0.berend@pobox.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary=oyUTqETQ0mS9luUI X-Mailer: Mutt 0.88 In-Reply-To: <01BD754F.A0AB9EF0.berend@pobox.com>; from Berend de Boer on Fri, May 01, 1998 at 10:22:28PM +0200 Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk --oyUTqETQ0mS9luUI Content-Type: text/plain; charset=us-ascii According to Berend de Boer: > My quite stable freebsd stable halted today with "swap_pager: out of swap > space". Only a reboot did help. But within minutes this message came again. > Help! I'm running stable from a few weeks ago. > > 1. What has happened? > > As far as I know someone send me a large (3MB) message, which sendmail > spooled to /var/spool/mqueue (at least there I found it) and procmail tried > to deliver it, without luck however. But /var/mail still did have 15MB > free, so what could be wrong? This is a 16MB machine with a 40MB swap > space. This is a well known problem with procmail on FreeBSD. The attached message contains a patch for procmail which solves the problem. -- Scott Kenney >|< saken@rmta.ml.org --oyUTqETQ0mS9luUI Content-Type: text/plain Content-Disposition: attachment; filename="msg00347.txt" _________________________________________________________________ [Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index][Thread Index][Top&Search][Original] _________________________________________________________________ Re: Procmail with huge messages (was Re: Can there be rules that only parse the headers?) _________________________________________________________________ * From: Andre Albsmeier * To: luomat+procmail@luomat.peak.org (Timothy J Luoma) * Cc: procmail@Informatik.RWTH-Aachen.DE * Date: Tue, 14 Oct 1997 22:31:34 +0200 (CEST) * Message-Id: <199710142031.WAA05368@curry.mchp.siemens.de> _________________________________________________________________ > Author: era eriksson > Original-Date: Mon, 13 Oct 1997 09:09:34 +0300 (EET DST) > Message-ID: <199710130609.JAA19818@kontti.Helsinki.FI> > > > I still think it's kind of embarrassing that Procmail would not be > > able to cope with big messages. If the normal local MDA could handle > > it, why can't Procmail? > > Procmail blew chunks all over a 9meg email I received this weekend. > > Sucked all available CPU and swapspace (about 250megs and a P-133). > > Got lots of duplicate emails and lost emails in the resultant disk crash(es) > which required manual fsck-ing. I had got this also on my FreeBSD machine with 64MB RAM and 250MB swap. No message bigger than 5-8 MB was possible. However, as I was told from the vm and malloc experts of FreeBSD, that this is due to the silly way how procmail reads messages: It always reallocs memory in 16k chunks until the message fits into (virtual) memory. With this info, I made a quick and dirty hack to procmail, and now it delivers 64MB messages on the same machine and that really quick. It now allocs 4MB chunks which, on a modern system, only get used if there is actually written to it. *** src/pipes.c.ORI Mon Apr 28 02:27:47 1997 --- src/pipes.c Fri Sep 19 13:23:18 1997 *************** *** 234,239 **** --- 234,240 ---- char*readdyn(bf,filled)char*bf;long*const filled; { int i;long oldsize; + long fetched = *filled; oldsize= *filled; goto jumpin; do *************** *** 243,249 **** if((size_t)*filled>=(size_t)(*filled+BLKSIZ)) lcking|=lck_MEMORY,nomemerr(); #endif ! bf=realloc(bf,*filled+BLKSIZ); /* dynamically adjust the buffer size */ jumpback:; } while(0<(i=rread(STDIN,bf+*filled,BLKSIZ))); /* read mail */ --- 244,253 ---- if((size_t)*filled>=(size_t)(*filled+BLKSIZ)) lcking|=lck_MEMORY,nomemerr(); #endif ! if( fetched - *filled < BLKSIZ ) { ! fetched += 4194304; ! bf=realloc(bf,fetched); /* dynamically adjust the buffer size */ ! } jumpback:; } while(0<(i=rread(STDIN,bf+*filled,BLKSIZ))); /* read mail */ -Andre _________________________________________________________________ References to: Timothy J Luoma _________________________________________________________________ [Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index][Thread Index][Top&Search][Original] _________________________________________________________________ --oyUTqETQ0mS9luUI-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message