Date: Mon, 09 Aug 1999 12:53:10 -0700 From: "Ronald F. Guilmette" <rfg@monkeys.com> To: Dan Nelson <dnelson@emsphone.com> Cc: questions@FreeBSD.ORG Subject: Re: Per-process memory overhead Message-ID: <19652.934228390@monkeys.com> In-Reply-To: Your message of Fri, 06 Aug 1999 10:38:40 -0500. <19990806103840.A68893@dan.emsphone.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I still have some unresolved questions about per-process space overhead.
See below.
In message <19990806103840.A68893@dan.emsphone.com>,
Dan Nelson <dnelson@emsphone.com> wrote:
>In the last episode (Aug 06), Ronald F. Guilmette said:
>> Can someone please explain to me why the following trivial program
>> shows up on both a `ps' listing and also when using `top' as having
>> a size of 136 KB?
>>
>> #include <unistd.h>
>>
>> int main (int argc, char *argv[]) { sleep (60); return 0; }
>>
>> Using the `size' command, the program itself, even when linked (with
>> the shared libraries) only has a trivial size of around 8 KB.
>>
>> So where the dickens is that other 128 KB of per-process overhead
>> coming from?
>
>Are you sure it's really per-process overhead? It's probably just part
>of libc or some other shared library. If I link your program static, I
>see an RSS of 20.
Thank you! Thank you! Thank you!
This message contained most of the hints I needed to mostly sort out the
problem.
My first problem was that I was looking at the VSZ (virtual size?) numbers,
rather than the RSS (resident set size?) numbers, as I should have been doing.
Also, after reading this message, I tried linking statically, and although
that alone didn't really reduce the RSS numbers for the program as much as
I would have hoped, it did lead me down a path towards being able to figure
out exactly where most of the per-process space overhead was coming from...
most of which I was then able to eliminate. (The program in question is
very simple, and it didn't really absolutely need to use stdio stuff, so I
managed to hack out all uses of stdio stuff and that saved quite a bit of
per-process space right there. More elimination of unneeded library calls
saved me even more space too.)
But I still have per-process space overhead problems... see below.
>> Note: This is NOT just an academic question. I need to run a LOT of
>> identical (small) processes, and this overhead is killing me bacuse I
>> really do not have enormous amounts of main memory available.
>
>If they really are identical, then you shuld simply worry about memory
>usage inside your program, since the memory pages holding the
>executable itself will be shared between all of the processes.
You are correct, of course. The executable code pages are all shared
between these many separate (process) instances of this program. So only
the data and stack space for each process are an issue.
As I said, the program is VERY small and simple. Now, I have it down to
the point where each instance of the program really only uses a single
4 KB page of memory dor data space and a single 4 KB page for stack space.
Swell.
So now each running instance of the program should really only take up
8 K bytes... after the executable code pages are factored out... right?
WRONG!
Each executing instance of this program now shows up as having an RSS of
36 KB. I _know_ that 12 KB of that is the (sharable) text segment, so that
leaves 24 KB, per process, to account for. Of that 24 KB, I already know
that the program itself is using 8 KB (for data and stack). But note that
this still leaves 16 KB per process unaccounted for.
So OK. Three questions:
1) Where is this extra 16 KB per-process overhead coming from?
2) How can I eliminate most of all of it?
3) Isn't a 16 KB per-process OS overhead a bit excessive?? I mean come on!
That is equal to 1/4 of all of the memory that was even available on the
original PDP-11s that first ran BSD UNIX! I really do think that this
amount of overhead, per process, may be difficult to justify.
So what is in that 16 KB anyway? Is this the legendary "U area" for the
process? If so, then I just gotta ask... Is the U area really that big??
I mean wow. I can fit a lot of stuff into 16 KB. Is BSD getting a bit
flabby in its old age? (1/2 :-)
If anyone can show me any way to get this per-process OS overhead whittled
down some, I would appreciate that greatly.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19652.934228390>