Date: Mon, 03 Apr 2006 14:38:36 +0200 From: Olaf Greve <o.greve@axis.nl> To: "Daniel A." <ldrada@gmail.com> Cc: freebsd-questions <freebsd-questions@freebsd.org> Subject: Re: How can I increase the shell's (or specific application's) memory limit? Message-ID: <4431174C.8020506@axis.nl> In-Reply-To: <5ceb5d550604030448t6ecf22uaef80f13f222c465@mail.gmail.com> References: <44310A0D.80607@axis.nl> <5ceb5d550604030448t6ecf22uaef80f13f222c465@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Daniel, > Generally, I think it's bad programming practice to retrieve such big > datasets if it is possible to do otherwise. I definitely agree that it is bad practice, and in that respect I'm inclined towards doing batch loading as you suggest too. However, there's some data agregation I'll have to take into account, and as it involves testing for the presence of specific tables of a merge table set, I'd have to rewrite part of that logic. All doable, of course, and no big issue either, but it would be a lot faster for me if I could simply increase the memory limit.... Still, I very much hear you, and I know that what you suggest _is_ the proper approach, so I may end up doing that too. ;) Also: there is another perhaps more elegant (read: robust) way, being a hybrid solution between the PHP script and using mysqldump. I can then use PHP for working out the batches, and retrieve the batches using a (set of) commandline mysqldump call(s). The generated batches can then directly be dumped into the proper merge tables. The only catch is that I directly left join data in into the merge tables, so I'd have to first do a blunt dump of the lhs of the data, then of the rhs(es) (both to temp tables) and then afterwards left join them into the eventual merge tables. This is the main reason why I hadn't chosen this solution, as at present I can combine all of these steps in one query...:/ If someone knows a clean way to increase the memory limit, I'd be happy to hear about it. If not, I'll do some rewrite... Cheers, Olafo
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4431174C.8020506>