Date: Fri, 05 Oct 2007 10:09:15 +0200 From: Jorn Argelo <jorn@wcborstel.com> To: Steve Bertrand <iaccounts@ibctech.ca> Cc: FreeBSD Questions <freebsd-questions@freebsd.org> Subject: Re: Managing very large files Message-ID: <4705F12B.1000501@wcborstel.com> In-Reply-To: <47054A1D.2000701@ibctech.ca> References: <4704DFF3.9040200@ibctech.ca> <200710041458.22743.wundram@beenic.net> <20071003200013.GD45244@demeter.hydra> <47054A1D.2000701@ibctech.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
Steve Bertrand wrote: >>> man 1 split >>> >>> (esp. -l) >>> >> That's probably the best option for a one-shot deal like this. On the >> other hand, Perl itself provides the ability to go through a file one >> line at a time, so you could just read a line, operate, write a line (to >> a new file) as needed, over and over, until you get through the whole >> file. >> >> The real problem would be reading the whole file into a variable (or even >> multiple variables) at once. >> > > This is what I am afraid of. Just out of curiosity, if I did try to read > the entire file into a Perl variable all at once, would the box panic, > or as the saying goes 'what could possibly go wrong'? > > Steve > Check out Tie::File on CPAN. This Perl module treats every line in a file as an array element, and the array element is loaded into memory when it's being requested. In other words: This will work great with huge files such as these, as not the entire file is loaded into memory at once. http://search.cpan.org/~mjd/Tie-File-0.96/lib/Tie/File.pm Jorn > _______________________________________________ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org" > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4705F12B.1000501>