Date: Mon, 25 Feb 2013 00:28:51 +0000 From: Ben Morrow <ben@morrow.me.uk> To: jdc@koitsu.org, freebsd-stable@freebsd.org Subject: Re: svn - but smaller? Message-ID: <20130225002847.GA42160@anubis.morrow.me.uk> In-Reply-To: <20130224212436.GA13670@icarus.home.lan> References: <1359380582-6256705.77592125.fr0SDgrYH000991@rs149.luxsci.com> <web-10502111@mailback3.g2host.com> <web-12014638@mailback4.g2host.com> <op.wszomvfyg7njmm@michael-think> <20130224031509.GA47838@icarus.home.lan> <op.wszrv9k5g7njmm@michael-think> <20130224041638.GA51493@icarus.home.lan> <op.wszt3wh2g7njmm@michael-think> <20130224063110.GA53348@icarus.home.lan> <1361726397.16937.4.camel@revolution.hippie.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
Quoth Jeremy Chadwick <jdc@koitsu.org>: > On Sun, Feb 24, 2013 at 10:19:57AM -0700, Ian Lepore wrote: > > On Sat, 2013-02-23 at 22:31 -0800, Jeremy Chadwick wrote: > > > > > > Also, John, please consider using malloc(3) instead of heap-allocated > > > buffers like file_buffer[6][] (196608 bytes) and command[] (32769 > > > bytes). I'm referring to this: > > > > Why? I absolutely do not understand why people are always objecting to > > large stack-allocated arrays in userland code (sometimes with the > > definition of "large" being as small as 2k for some folks). > > This is incredibly off-topic, but I'll bite. > > I should not have said "heap-allocated", I was actually referring to > statically-allocated. > > The issues as I see them: > > 1. Such buffers exist during the entire program's lifetime even if they > aren't actively used/needed by the program. With malloc(3) and friends, > you're allocating memory dynamically, and you can free(3) when done with > it, rather than just having a gigantic portion of memory allocated > sitting around potentially doing nothing. If the 'allocated' memory isn't touched, it will never be paged in. Even once it is paged in, if it then goes back to being unused it can be paged out to swap (when necessary) and then stay there. (It would be nice if there were some way to tell the system 'this memory is dead, don't write it out to swap', but I don't think there is.) Of course, you may get up to two pages of an object paged in just because some other object on the same page was touched, but 'two pages' is not a lot of memory to waste (especially given that you're saving the malloc overhead). I don't know if the compiler is clever enough to page-align large static allocations; that would reduce the potential waste to one page. > 2. If the length of the buffer exceeds the amount of stack space > available at the time, the program will run but the behaviour is unknown > (I know that on FreeBSD if it exceeds "stacksize" per resource limits, > the program segfaults at runtime). With malloc and friends you can > gracefully handle allocation failures. (SIGSEGV on stack overflow is specified by POSIX, including the fact that the signal must be fatal unless the signal handler uses an alternate stack.) Statically-allocated objects are not allocated on the stack, they are allocated in the bss or data sections of the executable. When it is possible and reasonable to do so, it's actually better to allocate all memory statically, as then once the process has started successfully you can be certain it won't fail because of a memory shortage. Large auto objects (including alloca) *are* dangerous. > 3. Statically-allocated buffers can't grow; meaning what you've > requested size-wise is all you get. Compare this to something that's > dynamic -- think a linked list containing pointers to malloc'd memory, > which can even be realloc(3)'d if needed. Under many circumstances a statically-allocated fixed-size buffer, *if properly used*, is safer than a potentially-unbounded malloced one. Of course, it's possible to put limits on the size of a malloced buffer as well, but (given that parts of the buffer you don't use won't ever be paged in) there isn't much advantage in doing so. Fixed allocations are only useful in small, single-use programs where you can accurately predict the memory requirements in advance. I wouldn't be surprised if svnsup fell into that bracket. Ben
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130225002847.GA42160>