From owner-freebsd-arch Tue Mar 25 9: 6:47 2003 Delivered-To: freebsd-arch@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 683) id 368B137B401; Tue, 25 Mar 2003 09:06:44 -0800 (PST) Date: Tue, 25 Mar 2003 09:06:43 -0800 From: Eivind Eklund To: David Schultz Cc: Garance A Drosihn , Dan Nelson , Poul-Henning Kamp , Wes Peters , freebsd-arch@FreeBSD.ORG Subject: Re: Patch to protect process from pageout killing Message-ID: <20030325090643.F20745@FreeBSD.org> References: <200303240823.48262.wes@softweyr.com> <7019.1048523782@critter.freebsd.dk> <20030324213519.GA63147@dan.emsphone.com> <20030325012844.GB4406@HAL9000.homeunix.com> <20030325075342.GA5450@HAL9000.homeunix.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20030325075342.GA5450@HAL9000.homeunix.com>; from das@FreeBSD.ORG on Mon, Mar 24, 2003 at 11:53:42PM -0800 X-Spam-Status: No, hits=-32.5 required=5.0 tests=EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES, REPLY_WITH_QUOTES,USER_AGENT_MUTT autolearn=ham version=2.50 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.50 (1.173-2003-02-20-exp) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Mar 24, 2003 at 11:53:42PM -0800, David Schultz wrote: > Yes, I think the SIGDANGER idea makes sense. Essentially what > you'd want is a higher threshhold above the ``red alert---start > killing things'' threshhold where you can do smart things like > send SIGDANGERs without worrying about running completely out of > memory. But I'm trying to impress on people that SIGDANGER is > orthogonal to what Wes is trying to do, before the whole thing > gets bogged down in discussions again and nothing ever happens. > Here's an example of what I mean in verbose pseudocode with > fudged constants: > > if (free VM < 64 pages) { > /* This is the part Wes is working on. */ > kill big processes EXCEPT the ones that > are so important that there's no point > in running the system without them; > } else if (free VM < 256 pages) { > /* > * It takes additional memory to do this, but we're > * hoping some processes will cooperate and the > * shortage will go away. > */ > start warning processes with SIGDANGER; > } As far as I understand, this problem was covered by the SIGDANGER proposal, by having processes with a SIGDANGER handler not be killed in the (free VM < 64 pages) case, at least until there are no processes without a SIGDANGER handler. The pseudo-code becomes something like (with 64, 128 and 256 being arbitrary constants) if (free VM < 256 pages) { send SIGDANGER to all processes } while (free VM < 128 pages && we have processes without SIGDANGER handler) { kill "worst" process without SIGDANGER handler } while (free VM < 64 pages) { /* * Only goes here if we are out of processes without * SIGDANGER handler */ kill "worst" process } As you see, SIGDANGER says that the process wants to decide for itself if it is important (kept until 64) or want to die/free up resources at 256. I'm not 100% happy with the SIGDANGER API, for the following reasons: (1) There are cases it does not cover. I can implement a process that is not really significant, but does caching and can easily free up memory. However, even though it frees up memory, it should not get special priority in the killing sequence. The most extreme example of this is if we add SIGDANGER awareness to phkmalloc - in that case, all newly compiled programs (that use libc and malloc) would gain priority, while all old binaries would be prioritized lower. (2) The use an API instead of an external configuration option (e.g. a sysctl with a list of protected PIDs) makes it impossible to use this without having recompiling software. It also means that priority is determined at the time of software implementation, not when the software is deployed, unless there are special options in the software to change behaviour. And these options are likely to appear, which basically sucks. However, I still feel that we *should* support the SIGDANGER API. We need an API to cover the case where a program has resources that it can easily release, and the API should be cross-platform. By supporting the SIGDANGER API on FreeBSD too, that API becomes aboutr 4x as "legitimate" as it is today. If we implement another API, both SIGDANGER and that API will be seen as less legitimate than SIGDANGER is today, unless that API is *much* better than SIGDANGER. Thus, we lower the chance of ever getting a true cross-platform solution. I feel there also is room for a separate solution that lets the administrator determine processes to keep - but this should not block for implementation of SIGDANGER with the standard semantics, and that is IMO what would be most important to have. Also note that a SIGDANGER implementation might automatically be picked up by autoconf for already existing programs, giving an immediate benefit. Eivind, who realize he has no vote until he has patches. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message