Date: Mon, 04 Feb 2008 01:52:15 +1030 From: Wayne Sierke <ws@au.dyndns.ws> To: Ed Schouten <ed@fxq.nl> Cc: Dag-Erling =?ISO-8859-1?Q?Sm=F8rgrav?= <des@des.no>, hackers@freebsd.org Subject: Re: sort(1) memory usage Message-ID: <1202052136.5350.137.camel@predator-ii.buffyverse> In-Reply-To: <20080203131322.GK1179@hoeg.nl> References: <8663x6mc2o.fsf@ds4.des.no> <20080203131322.GK1179@hoeg.nl>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 2008-02-03 at 14:13 +0100, Ed Schouten wrote: > * Dag-Erling Smørgrav <des@des.no> wrote: > > I've been trying to figure out why some periodic scripts consume so much > > memory. I've narrowed it down to sort(1). > > > > At first, I thought the scripts were using it inefficiently, feeding it > > more data than was really needed. Then I discovered this: > > > > des@ds4 ~% (sleep 10 | sort) & (sleep 5 ; top -o res | grep sort) > > [1] 66024 > > 66024 des 1 -8 5 54796K 52680K piperd 1 0:00 0.88% sort > > > > That's right - sort(1) consumes 50+ MB of memory doing *nothing*. > > > > (roughly half that on a 32-bit box) > > > > Something is rotten in the state of GNU... > > On my i386 box it spends 27M, but when I replace `sort' with `sed', > without any arguments, it's only 1.4 MB. I tried this on RELENG_6. I can > also reproduce this on Linux. > %uname -vm FreeBSD 7.0-PRERELEASE #1: Fri Jan 25 01:08:47 CST 2008 root@freebsd7stable-1.vmware:/usr/obj/usr/src/sys/GENERIC-KTR-0x2000 i386 %( sleep 10 | sort ) & ( sleep 5 ; top -n 150 | grep sort ) [2] 38158 38158 ws 1 -8 0 29760K 736K piperd 0:00 0.00% sort %su - # ( sleep 10 | sort ) & ( sleep 5 ; top -n 150 | grep sort ) [2] 38165 38165 root 1 -8 0 29760K 732K piperd 0:00 0.00% sort $ uname -vm FreeBSD 6.3-PRERELEASE #1: Fri Dec 28 17:49:43 CST 2007 root@predator-ii.buffyverse:/usr/obj/usr/src/sys/LILLITH-IV i386 $ (sleep 10 | sort) & (sleep 5; top -n 150 | grep sort) 68953 ws 1 -8 0 26988K 660K piperd 0:00 0.00% sort $ su - # (sleep 10 | sort) & (sleep 5; top -n 150 | grep sort) [1] 68981 68981 root 1 -8 0 26988K 660K piperd 0:00 0.00% sort Next one is Ubuntu 7.04 $ uname -a Linux developer 2.6.20-16-generic #2 SMP Thu Jun 7 20:19:32 UTC 2007 i686 GNU/Linux $ (sleep 10 | sort) & (sleep 5; ps aux | grep -E "^USER|sort$") [9] 10523 USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND ws 10526 0.0 0.0 28436 676 pts/4 S 01:29 0:00 sort (I had to change the 'top' incantation to pick up the 'sort' process on these busy boxes.) Wayne
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1202052136.5350.137.camel>