From owner-freebsd-questions@FreeBSD.ORG Mon Oct 2 19:08:01 2006 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E90A016A412 for ; Mon, 2 Oct 2006 19:08:01 +0000 (UTC) (envelope-from SRS0=pfkq1ow3=DP=asarian-host.net=admin@asarian-host.net) Received: from mail.asarian-host.net (mail.asarian-host.net [194.109.160.70]) by mx1.FreeBSD.org (Postfix) with ESMTP id 65A4943D60 for ; Mon, 2 Oct 2006 19:07:56 +0000 (GMT) (envelope-from SRS0=pfkq1ow3=DP=asarian-host.net=admin@asarian-host.net) Received: from anonymizer.asarian-host.net (localhost [127.0.0.1]) (authenticated bits=0) by mail.asarian-host.net (8.13.5/8.13.5) with ESMTP id k92J7seL029506 for ; Mon, 2 Oct 2006 21:07:54 +0200 (CEST) (envelope-from admin@asarian-host.net) From: Mark Received-SPF: pass (asarian-host.net: 127.0.0.1 is authenticated by a trusted mechanism) receiver=anonymizer.asarian-host.net; client-ip=127.0.0.1; envelope-from=; helo=clientrunner.asarian-host.net; Message-Id: <200610021907.k92J7pxs029496@asarian-host.net> Date: Mon, 02 Oct 2006 19:07:54 GMT X-Authenticated-Sender: admin@asarian-host.net X-Trace: cmXlo/lVhQixGyyszPoEcDos0FypLe4YacVpxAE6Lfn/c5buVcS8TMonW2tqwhPY X-Complaints-To: abuse@asarian-host.net X-Abuse-Info: Please be sure to forward a copy of ALL headers, otherwise we are unable to process your complaint Organization: Asarian-host To: "'FreeBSD-Questions Questions'" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2962 In-Reply-To: <4520EEE5.5010409@dial.pipex.com> X-Auth: Asarian-host PGP signature iQEVAwUARSFjijFqW1BleBN9AQH8egf/fVarYqmcnMLej+3riocEsqsE2/expikS AJkQgUvE3DxGaYoCL55KvFtTBvMYO7GtwPFsUmO33stVtbcyiVqwnjaAOieuIe4T LY26XYKlODggIlvcGvU5t5rGKxJ6NLHjUs6/0CrlAU3RaBLx1ekP8ukhcvM6GhLN 0yrMjRmaAB1D2Jyf61wKX9LeH17PopopR14hWG6W4TLxHXyXBaGrkpcTdwhy6STg nBDEK1I5CWKmu5C+ndekEIqA5zL9aVaohLJHgGmz6c4RfWN4Z2LyOJJGHytVAOQ9 kWJXODL/ZnjkpVZ0H4WPH3PVN8m/mtZaSBuxk3OoaBnkjG4hplMNAQ== =spGj Subject: RE: Trouble with Berkeley DB version 4.4.20? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Oct 2006 19:08:02 -0000 > -----Original Message----- > From: Alex Zbyslaw [mailto:xfb52@dial.pipex.com] > Sent: maandag 2 oktober 2006 12:50 > To: Mark > Cc: 'FreeBSD-Questions Questions' > Subject: Re: Trouble with Berkeley DB version 4.4.20? > > Thanks for answering, again. > > I actually already tried p5-BerkeleyDB too. Had high hopes for it. It > > took a good 10 minutes or so to compile a new gcc, 3.2.23, first; but > > after that, everything went well. Except that it shows the same behav- > > ior. Compile goes fine; "make tests" too. But my app core dumps every, > > say, 30 times it accesses BerkeleyDB. > > Not sure why p5-BerkeleyDB thinks it needs a specific gcc, though > 3.2.23 is the version I have anyway (FreeBSD 5.4) so I didn't know to > warn you :-( For all the difference it made. :) > I don't know what your apps do, but make sure for example, that if you > can have multiple instances which can write to the db at the same time > that your are locking correctly. In 1.85 compatibility usage that means > locking a separate file with e.g. flock and *not* locking the database > file itself which doesn't work right. I know. In fact, I have one mini-Perl installed, with a DB_File compiled against BerkeleyDB 1.85, for that precise reason (a Perl process that shares the db with DRAC). > PS I know there isn't a port yet, but you could just try compiling 4.5 > from the sleepycat sources. The port for 4.4 doesn't look like it does > anything sophisticated: just applies the latest patches from sleepy- > cat, sets up some configure args, and has one small patch to dist/con- > figure which changes -avoid-version into -version-info 0:0:0 in a cou- > ple of places. Already did that; I got impatient. :) The changes to the Makefile are minor, and I got 4.5.20 to compile and install just great. > Waiting for 4.5 is an option, but I would worry that whatever change in > 4.4 is making your app crash might also bite in 4.5. I could find noth- > ing on google about perl+4.4 core dumping so I would harbour a suspicion > that it is something about the way you are using BerkeleyDB which is > causing the trouble. Upgrading to 4.5 made no difference. I even installed an entirely new Perl (5.8.8) for the occasion. To no avail. I really suspect it's a locking issue. My app is a socketmap daemon, for sendmail, that does SPF queries and the like. So, concurrency is definitely happening when processes are forking. The one time I was actually able to log an error for the process, it said: "Invalid locker id" when opening the BerkeleyDB environment. That in itself may not necessarily mean anything (could just mean the env has become corrupted). But the occurance of the core dumps are consistent with a locking issue (under heavy, concurrent load). BerkeleyDB 0.30 (the Perl package) itself takes care of locking the shared environment. Paul Marquess, the author of BerkeleyDB (the Perl package), wrote me once: "Apart from that you don't need to do anything -- Berkeley DB handles the locking for you behind the scenes." And it's certainly true for for 4.2: the 'tied' hashes can be accessed, concurrently, between processes, even under the most stringent load. Since an upgrade to 4.5 made no difference, I'm really thinking this is just a matter of BerkeleyDB 0.30 (the Perl package) no longer doing the locking properly. Or maybe there's something between BerkeleyDB 4.2 and 4.4 that has changed about the manner in which locks are done that BerkeleyDB 0.30 is unaware of. - Mark