Date: Thu, 30 Nov 2006 10:32:18 +0000 (GMT) From: Robert Watson <rwatson@FreeBSD.org> To: Kris Kennaway <kris@obsecurity.org> Cc: freebsd-stable@freebsd.org, "van Osnabrugge, Sean" <svanosnabrugge@osler.com> Subject: Re: grep: memory exhausted Message-ID: <20061130103025.N95096@fledge.watson.org> In-Reply-To: <20061130044123.GA19761@xor.obsecurity.org> References: <7883F37393F1D44990888174C2E5E91222666A@TSV-EXCHANGE.ohhllp.com> <20061130044123.GA19761@xor.obsecurity.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 29 Nov 2006, Kris Kennaway wrote: > On Wed, Nov 29, 2006 at 03:10:33PM -0500, van Osnabrugge, Sean wrote: >> >> I am running a fresh install of FreeBSD-6.1-Stable as a guest OS in VMWare >> 1.0.1 with 1 GB of RAM. >> >> Whenever I try to grep a large text file (400 MB+), grep terminates with >> "grep: memory exhausted" >> >> I have tried piping grep (cat "file" | grep "search term") >> >> I have tried it with -line-buffered >> >> ulimit -a show: >> >> core file size (blocks, -c) unlimited >> >> data seg size (kbytes, -d) 524288 > > Try increasing this. I think grep mmaps the file, so the large file could > be exceeding your limit. I regularly grep multi-hundred-gigabyte files without a problem on boxes with very little memory, so simple file size is unlikely the cause. However, it would be interesting to know how many lines "wc -l" thinks the text file has, and what the length of the longest line is. Robert N M Watson Computer Laboratory University of Cambridge
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20061130103025.N95096>