From owner-freebsd-questions@FreeBSD.ORG Sun Mar 15 16:31:16 2009 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93FF91065675 for ; Sun, 15 Mar 2009 16:31:16 +0000 (UTC) (envelope-from cpghost@cordula.ws) Received: from fw.farid-hajji.net (fw.farid-hajji.net [213.146.115.42]) by mx1.freebsd.org (Postfix) with ESMTP id EF4488FC1A for ; Sun, 15 Mar 2009 16:31:15 +0000 (UTC) (envelope-from cpghost@cordula.ws) Received: from phenom.cordula.ws (phenom [192.168.254.60]) by fw.farid-hajji.net (Postfix) with ESMTP id A9A8A326BE; Sun, 15 Mar 2009 17:31:13 +0100 (CET) Date: Sun, 15 Mar 2009 17:31:13 +0100 From: cpghost To: Dan Nelson Message-ID: <20090315163113.GA1044@phenom.cordula.ws> References: <20090315091301.GB1051@phenom.cordula.ws> <20090315154849.GE24875@dan.emsphone.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20090315154849.GE24875@dan.emsphone.com> User-Agent: Mutt/1.5.19 (2009-01-05) Cc: freebsd-questions@freebsd.org Subject: Re: Speeding up exit(2)? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Mar 2009 16:31:17 -0000 On Sun, Mar 15, 2009 at 10:48:49AM -0500, Dan Nelson wrote: > In the last episode (Mar 15), cpghost said: > > I've noticed that when a huge, partially or totally swapped out process > > exits, there is a lot of disk activity going on, before the process truly > > dies. This is not necessarily due to sync(2), because it also happens > > with CPU bound processes that write very little output. > > > > Not sure what's really going on there, but apparently, the process reads > > in pages from swap that have been paged out previously (according to > > top(1)). > > Are you sure this is actually in _exit, and not in a cleanup function > executed by the application as it exits? If there is a large linked list, > for example, and the author has decided to actually free the list before > exiting instead of just letting it disappear when the process exits, each > swapped-out page will have to be brought back in as the list is traversed. > C++ programs may have destructors doing this behind the scenes. Yes, that's quite possible. Meanwhile, I'm suspecting that free(3) is the culprit, and not the vm subsystem itself; though I was not yet able to construct a good example to be sure. And you're quite right: cleaning up paged-out linked lists or other dynamic data structures, either explicitly or via C++ destructors is also an obvious reason for swap activity. Didn't think of it first. > Best way to figure out what's going on is to attach to the program with gdb > while it's swapping, and print a stack trace. I'll try this. Very good idea. > Also, since you mentioned a "totally swapped out" process exiting, are you > terminating it externally with kill -9? It may be writing a core dump, > which will force the kernel to pull back swapped-out pages to write them to > the core file. Also an excellent point. I'm just killing them with Ctrl-C (SIGINT), which won't result in a core dump. But a core dump would also reawaken the pages, that's quite clear. > Dan Nelson > dnelson@allantgroup.com Thanks for all the hints. The fog is slowly lifting. ;-) -cpghost. -- Cordula's Web. http://www.cordula.ws/