Date: Wed, 14 Jul 1999 14:05:20 -0700 (PDT) From: Matthew Dillon <dillon@apollo.backplane.com> To: Robert Elz <kre@munnari.OZ.AU> Cc: "Daniel C. Sobral" <dcs@newsguy.com>, freebsd-hackers@FreeBSD.ORG, tech-userlevel@netbsd.org Subject: Re: Replacement for grep(1) (part 2) Message-ID: <199907142105.OAA96038@apollo.backplane.com> References: <20351.931984551@cs.mu.OZ.AU>
next in thread | previous in thread | raw e-mail | index | archive | help
:Now let's look at what happens with the two methods.
:
:With all VM backed by real mem or swap space, processes go about allocating
:memory - when there is no more left, the allocations start failing.
:If the process is perl, it just collapses in a heap, and the log file
:summary doesn't get made that day. So sad... If its sendmail, it
:issues "OS error, temporary failure" type responses, saves its queue files,
:and exits. A later sendmail will deliver those messages, no harm.
:If its a shell, who knows (I forget what the shells do, I think most just
:keep trying, at least if interactive), but they consume mem at such a slow
:rate it doesn't matter - fork() would typically fail though, so no new
:processes could get started. innd would just pause, and wait till a
:bit later when mem might be available again (those perls and sendmails
:all gone away). named just the same (at least the named munnari ran).
:They're the two processes munnari was supposed to be runinng - those two
:don't just die.
Which means that if one of those two processes happen to be the ones
primarily responsible for running the machine out of VM, memory resources
will never be released and now you can't even login! Not only that, but
if you are running a news subsystem, it is actually *worse* if the news
process bogs down and gets behind then it for the news process to simply
die and alert someone. When you are pushing news, you cannot afford to
get behind.
Also, your named is badly misconfigured if it grows to 130MB. We never
allow ours to grow past 30MB.
Since the machine is basically in an unworking state anyway, and since
you can now no longer login, I don't quite see why you are happy that
those two processes are still running. From my standpoint, the machine
is badly broken and needs to be rebooted and then fixed so the problems
do not reoccur and I would be much happier if I could log into the beast
to get that done then to have to hit the reset button.
:Now, with overcommit mode, we get an extra 30 seconds of life, because
:no doubt there are a few pages floating around that have been allocated
:to some process, but nothing has bothered to write into yet. An extra 30
:... garbage removed ...
:Sure it would get lots of VM back again, but the system would no longer
:have been doing what it was supposed to be doing. Adding more swap space
The machine isn't doing what it is supposed to be doing in either case
once it has run out of VM. Except in the first case you think you should
be happy because it didn't kill the news process, when in fact you ought
to be trying to figure out why the thing ran out of VM in the first place
and then fix it so it never happens again.
To me, this whole scenario sounds like a badly configured machine which
the sysop isn't willing to fix. I feel sorry for the poor company who
hired that sysop!
:would be easy, but the wrong thing to do, that would just have allowed
:the system to page itself to death, thrashing into eternity - having
:processes go away is the only solution to this kind of problem. Except
:it needs to be the right processes, and "right" does not equal "big",
:nor any other criteria the kernel could possibly figure out for itself.
:
:kre
If you consider this a critical problem, then the only acceptable solution
is to write a watchdog script that monitors swap utilization and kills
the correct processes if swap starts to get low. If you wait until swap
actually runs out, you've already lost because too many things are likely
to break in a general purpose computing environment. Of course I suppose
you could advocate that programs must be written 'properly' to handle
the case... well, more power to you, but in a general computing environment
you are running dozens if not hundreds of third party applications and
fixing them all is a pipe dream.
It seems to me that you are willing to blame the operating system for
a situation that is really not the OS's fault, and that you are not willing
to sit down and spend the 10 minutes necessary writing a simple watchdog
script.
I don't bother to write watchdog scripts to check for swap, because my
machines DO NOT RUN OUT OF SWAP. If your machines do, then maybe you
should consider writing the watchdog script. Personally, I think you would
get better reliability by fixing your systems.
You are blaming what is essentially a last-resort effort by the kernel for
not being nice to your processes. Well Duh! It's a last-resort mechanism,
it isn't supposed to be nice. Maybe you shouldn't be depending on last
resort mechanisms to keep your machines running.
-Matt
Matthew Dillon
<dillon@backplane.com>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199907142105.OAA96038>