From owner-freebsd-current@FreeBSD.ORG  Sun May 25 19:40:03 2003
Return-Path: <owner-freebsd-current@FreeBSD.ORG>
Delivered-To: freebsd-current@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id D54B837B401
	for <freebsd-current@freebsd.org>;
	Sun, 25 May 2003 19:40:02 -0700 (PDT)
Received: from smtp01.syd.iprimus.net.au (smtp01.syd.iprimus.net.au
	[210.50.30.52])	by mx1.FreeBSD.org (Postfix) with ESMTP id E530643F75
	for <freebsd-current@freebsd.org>;
	Sun, 25 May 2003 19:40:01 -0700 (PDT)
	(envelope-from tim@robbins.dropbear.id.au)
Received: from dilbert.robbins.dropbear.id.au (210.50.252.147) by
	smtp01.syd.iprimus.net.au (7.0.015)
	id 3ECDA4450005D74E; Mon, 26 May 2003 12:39:58 +1000
Received: by dilbert.robbins.dropbear.id.au (Postfix, from userid 1000)
	id 39208C90F; Mon, 26 May 2003 12:39:54 +1000 (EST)
Date: Mon, 26 May 2003 12:39:53 +1000
From: Tim Robbins <tjr@FreeBSD.ORG>
To: Anthony Schneider <anthony@x-anthony.com>
Message-ID: <20030526123953.A91078@dilbert.robbins.dropbear.id.au>
References: <20030525064929.GA96588@x-anthony.com>
	<20030525211730.GA5226@x-anthony.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.2.5.1i
In-Reply-To: <20030525211730.GA5226@x-anthony.com>;
	from anthony@x-anthony.com on Sun, May 25, 2003 at 05:17:30PM -0400
cc: freebsd-current@freebsd.org
Subject: Re: mpi + shmem issues
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
	<freebsd-current.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
	<mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
	<mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 26 May 2003 02:40:03 -0000

On Sun, May 25, 2003 at 05:17:30PM -0400, Anthony Schneider wrote:

> so, does this mean that mpich somehow exhausted all shmem locks?
> after running the program only 10 times, i see this as infeasible,
> considering
> 	a) mpich (presumably in MPI_Init()) would only want 1 or
> 	   2 locks on init
> and
> 	b) any shared memory locks mpich grabs should be freed
> 	   upon process completion (whether clean or not) by the
> 	   operating system, no?

No, semaphores stay around until they are removed. Perhaps there's an MPI_Xyz
function you should be calling before your program exits. In any case, you can
use "ipcs -s" to list semaphores, and "ipcrm -s semid" to remove semaphores.

It looks like the program is leaking semaphores -- notice how your program
fails when you run it for the 11th time, and the default maximum number of
semaphores in the GENERIC kernel is 10 (sysctl kern.ipc.semmni).


Tim