From owner-freebsd-database@FreeBSD.ORG Mon Oct 10 18:00:10 2005 Return-Path: X-Original-To: freebsd-database@freebsd.org Delivered-To: freebsd-database@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2E2CF16A447 for ; Mon, 10 Oct 2005 18:00:10 +0000 (GMT) (envelope-from mikes@siralan.org) Received: from rader.servnow.com (rader.servnow.com [69.93.129.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8DA4043D46 for ; Mon, 10 Oct 2005 18:00:09 +0000 (GMT) (envelope-from mikes@siralan.org) Received: from 12-210-221-89.client.insightbb.com ([12.210.221.89] helo=familysquires.net) by rader.servnow.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.52) id 1EP1wR-0002GV-9Z for freebsd-database@freebsd.org; Mon, 10 Oct 2005 14:00:03 -0400 Date: Mon, 10 Oct 2005 13:00:07 -0500 (EST) From: "Michael L. Squires" X-X-Sender: mikes@familysquires.net To: freebsd-database@freebsd.org In-Reply-To: <20051005005957.66798.qmail@web32910.mail.mud.yahoo.com> Message-ID: <20051010125401.P81838@familysquires.net> References: <20051005005957.66798.qmail@web32910.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - rader.servnow.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - siralan.org X-Source: X-Source-Args: X-Source-Dir: Subject: Oracle 10g r2 Instant Client X-BeenThere: freebsd-database@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Database use and development under FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2005 18:00:10 -0000 I'm playing with the 10g release 2 LINUX Instant Client using the 10g r1 ports installation as a base. I hope to connect to a 10g r1 server at work. At this point the basic and SQL*Plus stuff installs. I have the SUSE 9.3 linux emulation installed; sqlplus can't find its own library, which is to be expected given that I haven't yet set up the library paths, ORACLE_HOME, or TNSNAMES.ORA. If someone has already done all of this, please advise; otherwise I'll post any successes/failures here as they happen. One package I'd like to try is TORA. The system on which things are installed is a Toshiba 8100 running 5.4-STABLE, Xorg, KDE, and eventually OpenOffice 2.0. Mike Squires mikes@siralan.org msquires@iga.state.in.us From owner-freebsd-database@FreeBSD.ORG Mon Oct 10 23:46:11 2005 Return-Path: X-Original-To: freebsd-database@freebsd.org Delivered-To: freebsd-database@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A47A616A420 for ; Mon, 10 Oct 2005 23:46:11 +0000 (GMT) (envelope-from drumslayer2@yahoo.com) Received: from web32907.mail.mud.yahoo.com (web32907.mail.mud.yahoo.com [68.142.206.54]) by mx1.FreeBSD.org (Postfix) with SMTP id 8986743D48 for ; Mon, 10 Oct 2005 23:46:10 +0000 (GMT) (envelope-from drumslayer2@yahoo.com) Received: (qmail 71652 invoked by uid 60001); 10 Oct 2005 23:46:10 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=LJVT/6h7sSf3wMR2sqkQD0GMd6Lloofw2I9983bnwfJSBC/+1iRwfW+e4wHALml7RdqJ0RWX2KxIQePLWqFyiyPGo30zAr94aZtCO2UMoi4+cnnmz4L8geQAaZyIRKC/sOx9GejX2EVG1oSdivXv+9ZCvhu5a5BseO3ZVgppg5E= ; Message-ID: <20051010234610.71650.qmail@web32907.mail.mud.yahoo.com> Received: from [67.112.21.27] by web32907.mail.mud.yahoo.com via HTTP; Mon, 10 Oct 2005 16:46:10 PDT Date: Mon, 10 Oct 2005 16:46:10 -0700 (PDT) From: NMH To: questions , FreeBSD-DataBase In-Reply-To: <20051005005957.66798.qmail@web32910.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: Mysql Dying at random times - points to a lib issue? - sollved Sortof X-BeenThere: freebsd-database@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Database use and development under FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2005 23:46:11 -0000 Just to follow up to my own post (and adding FreeBSD Questions)- I wanted to share that oddly enough, going backwards and installing FreeBSD 4.11 with Mysql 4.11 With LinuxThreads solved the problem of the database dying! I would love any opinions on why this could be. It seems to have found a possible SCSI issue, but so far MYSQL hasn't died in 36 hrs. it usualy was at least once every 8-12. Thanks NMH The hardware.. Tyan S2882 MB with one Opteron 248 - 4 gigs of ram Was FreeBSD 5.4-RELEASE-p6 i386 Now FreeBSD 4.11-STABLE i386 Infortrend RAID unit with 4 channels. 3 channels of RAID 1 storage with heavy databases tables on seperate arrays. (symlinked) and one channel for binlogs. --- NMH wrote: > > Hello.. > > Currently, our mysql database is crashing at random > intervals though out the day during heavy load. A > mysql support engineer traced the problem to > libthr. > > > #0 0x28619dfe in mutex_self_lock (mutex=0x0, > noblock=0) at > /usr/src/lib/libthr/thread/thr_mutex.c:615 > 615 PANIC("Shouldn't resume here?\n"); > > The Database server is running i386 FreeBSD 5.4 > Release-P6, with 4 gigs of ram, on an > AMD Opteron(tm) Processor 248 (2191.84-MHz 686-class > CPU). W/ infotrend raiding solution. > > Apparently, there have been some updates (beyond my > ability to understand) to the libthr package, which > have not yet been backported to the 5.4 > release...?? (changes in 6.0 that seem pertinant) > > We have attempted to use KSE, and pthreads Binary > from MYSQL, but we encountered even WORSE problems > with those packages. > The mysqld would consistantly crash in those > threading > packages, usually within a minute or so of > testing... > Usually with a message > that a junk pointer was being released. (Error came > from malloc.c). > > We were able to reproduce the memory problem on a > seperate machine with a different hardware > configuration, but same OS. And on other machines > with the same hardware configuration. > > > A mysql support engineer is currently checking > around > to see if there are any solutions, but meanwhile... > I > thought many heads are better than one. > > All of these problems started when we migrated to > this new server, which hardware wise was very > similiar > to the old one. However the old one was running > Mysql > 4.0.18 and FreeBSD 5.1-Release-P11. > According to /usr/src/UPDATING some lib changes > occured after this time. > > Currenly, on their suggestion, we are using > /etc/libmap.conf entry of > libpthread.so libthr.so > libpthread.co.1 libthr.so.1 > > Which seems backwards to what people on here have > recommended. But seemed to help but we are still > crashing at least once a day sometimes more. > > > Any pointers, tips etc quite welcomed. > > Thanks! > The Large Print Giveth And The Small Print Taketh Away -- Anon __________________________________ Yahoo! Music Unlimited Access over 1 million songs. Try it free. http://music.yahoo.com/unlimited/ From owner-freebsd-database@FreeBSD.ORG Wed Oct 12 06:18:47 2005 Return-Path: X-Original-To: freebsd-database@freebsd.org Delivered-To: freebsd-database@freebsd.org Received: from hub.freebsd.org (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id EF48616A425 for ; Wed, 12 Oct 2005 06:18:47 +0000 (GMT) (envelope-from owner-moderators@freebsd.org) MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit From: owner-moderators@freebsd.org To: freebsd-database@freebsd.org Message-ID: Date: Tue, 11 Oct 2005 23:18:46 -0700 Precedence: bulk X-BeenThere: moderators@freebsd.org X-Mailman-Version: 2.1.5 X-List-Administrivia: yes Sender: owner-moderators@freebsd.org Errors-To: owner-moderators@freebsd.org Subject: Your message to moderators awaits moderator approval X-BeenThere: freebsd-database@freebsd.org List-Id: Database use and development under FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2005 06:18:48 -0000 Your mail to 'moderators' with the subject hi Is being held until the list moderator can review it for approval. The reason it is being held: SpamAssassin identified this message as possible spam Either the message will get posted to the list, or you will receive notification of the moderator's decision. If you would like to cancel this posting, please visit the following URL: http://lists.freebsd.org/mailman/confirm/moderators/00d9f305bf43e5f5dc31959618f165ab159da781 PLEASE NOTE! If you would like to post freely to the list, please subscribe first. If you post from multiple addresses, you can subscribe each address and go into the options page and select 'no mail' for all but one address. This will allow you to post without delay in the future. Sorry for the hassle, but certain immature people made this necessary. From owner-freebsd-database@FreeBSD.ORG Wed Oct 12 20:16:24 2005 Return-Path: X-Original-To: freebsd-database@freebsd.org Delivered-To: freebsd-database@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 035CC16A41F for ; Wed, 12 Oct 2005 20:16:24 +0000 (GMT) (envelope-from drumslayer2@yahoo.com) Received: from web32905.mail.mud.yahoo.com (web32905.mail.mud.yahoo.com [68.142.206.52]) by mx1.FreeBSD.org (Postfix) with SMTP id 546AD43D49 for ; Wed, 12 Oct 2005 20:16:23 +0000 (GMT) (envelope-from drumslayer2@yahoo.com) Received: (qmail 49706 invoked by uid 60001); 12 Oct 2005 20:16:22 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=zyY56kQSHKK09hoW4qYKXRk05xgn03ZlsLq46UEXgdfT89CwJfapAMPdKnmApWKTJw8mOihmvrWKT70AifYtGUDo3pK54vMUUlMeUYgCLhAFXb+NynNoB4zcPq1OuLownLtNtnHvpvdOY9tl2SUJdoxKtMB8x9DuSShb6voTi1c= ; Message-ID: <20051012201622.49704.qmail@web32905.mail.mud.yahoo.com> Received: from [67.112.21.27] by web32905.mail.mud.yahoo.com via HTTP; Wed, 12 Oct 2005 13:16:22 PDT Date: Wed, 12 Oct 2005 13:16:22 -0700 (PDT) From: NMH To: FreeBSD-DataBase , questions MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Mysql server not able to stay running on anything but Linux? X-BeenThere: freebsd-database@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Database use and development under FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2005 20:16:24 -0000 Hi I am stuck with a delema and I feel like a damn troll. But.. I have a Mysql Database that I posted about earlier. It seems that it is only able to not die by running on BSD 4.11 with Linux Threads. My boss is convinced this means that Linux is better for MySQL and wants that installed now. We even got a support contact from Mysql that so far has gotten us nothing for almost a month while our production database server died up to 3 times a day. (and lots of we're looking into it's) We were running fine but a little slow on FreeBSD 5.1-P11 and MySQL 4.0.18.(apperantly before a big Lib change) We had to move quicker than we wanted to a new server running FreeBSD 5.4 and MySQL 4.11 (becouse of a dual HD death) Under production load the new 5.4 server fell over regulary. It has only now become stable by wiping it and running it on FreeBSD 4.11 with Linux Threads. (it regularly has over 400+ threads) MySQL hinted to us that for better support we should move to Linux since that is what they develop on and would somehow better help them solve the problem. I really don't want to run Linux! We have been a FreeBSD only shop for some time and I find it amazing that 5.4 BSD can't support Mysql under a heavy 400+ thread load without Mysql falling over itself and dying. I want to try FreeBSD 5.4 AMD64 (the machines are Opteron) or 6.0 but my boss feels that would be a waste given that MYSQL doesn't support Mysql on AMD64 well enough. Also that given their hints that for the "best" support we should move our database to Linux to help solve this issue and that so far it has only run with any stability using Linux Threads, So why not just run Linux. That should be even better. Can anyone help or offer assistance to help track this down? Perhaps also any annecdotes or examples I can show my boss that other people have as busy MYSQL databases on BSD 5.X. We paid 3K to Mysql for help and so far they have been unable to offer any clues as to why ours will not stay stable on anything but Linux threads. I feel really sad that so far no one has responded to my posts and it feels like a victory for linux. As I say I only manage the server, I don't program the databases. Is there anything I should/could look for database wise that could trigger such things? The biggest problem with all this is that we have not been able to replicate it. The Mysql Server on 5.4 only dies when under production load. Nothing we or Mysql have been able to simulate seems to have the same effect. Any idea why this could be as well? Not enough randomness? Please help me keep a Freebsd only shop that way. Thanks! NMH The Large Print Giveth And The Small Print Taketh Away -- Anon __________________________________ Yahoo! Music Unlimited Access over 1 million songs. Try it free. http://music.yahoo.com/unlimited/ From owner-freebsd-database@FreeBSD.ORG Wed Oct 12 21:08:07 2005 Return-Path: X-Original-To: freebsd-database@freebsd.org Delivered-To: freebsd-database@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 66F0916A41F; Wed, 12 Oct 2005 21:08:07 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9750543D6A; Wed, 12 Oct 2005 21:08:00 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 54EA71A3C24; Wed, 12 Oct 2005 14:08:00 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 2DC505132E; Wed, 12 Oct 2005 17:07:58 -0400 (EDT) Date: Wed, 12 Oct 2005 17:07:57 -0400 From: Kris Kennaway To: NMH Message-ID: <20051012210757.GA97184@xor.obsecurity.org> References: <20051012201622.49704.qmail@web32905.mail.mud.yahoo.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wac7ysb48OaltWcw" Content-Disposition: inline In-Reply-To: <20051012201622.49704.qmail@web32905.mail.mud.yahoo.com> User-Agent: Mutt/1.4.2.1i Cc: FreeBSD-DataBase , questions Subject: Re: Mysql server not able to stay running on anything but Linux? X-BeenThere: freebsd-database@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Database use and development under FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2005 21:08:07 -0000 --wac7ysb48OaltWcw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Oct 12, 2005 at 01:16:22PM -0700, NMH wrote: > Hi > I am stuck with a delema and I feel like a damn > troll. But.. I have a Mysql Database that I posted > about earlier. It seems that it is only able to not > die by running on BSD 4.11 with Linux Threads. My boss > is convinced this means that Linux is better for MySQL > and wants that installed now. > We even got a support contact from Mysql that so far > has gotten us nothing for almost a month while our > production database server died up to 3 times a day. > (and lots of we're looking into it's) > =20 > We were running fine but a little slow on FreeBSD > 5.1-P11 and MySQL 4.0.18.(apperantly before a big Lib > change) We had to move quicker than we wanted to a new > server running FreeBSD 5.4 and MySQL 4.11 (becouse of > a dual HD death) Under production load the new 5.4 > server fell over regulary. It has only now become > stable by wiping it and running it on FreeBSD 4.11 > with Linux Threads. (it regularly has over 400+ > threads) Unfortunately you'll need to provide details of how it "fell over" (e.g. panic messages + backtraces). You might have better luck with 6.0-RELEASE or 5.4-STABLE, since a number of bugs have been fixed in 5.4-RELEASE..but without more details it's impossible to say whether that will help, or if you're encountering something genuinely new. > Can anyone help or offer assistance to help track > this down? Perhaps also any annecdotes or examples I > can show my boss that other people have as busy MYSQL > databases on BSD 5.X. We paid 3K to Mysql for help and > so far they have been unable to offer any clues as to > why ours will not stay stable on anything but Linux > threads. I feel really sad that so far no one has > responded to my posts and it feels like a victory for > linux.=20 If I was your boss I'd be asking why mysql hasn't delivered on their support contract. Kris --wac7ysb48OaltWcw Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFDTXssWry0BWjoQKURAlzeAKDW0m8IVWSGepPVTYbnvDtP9p7pPQCguIBa 91ehGxFGTyfWLwqaGyaB03k= =wYZi -----END PGP SIGNATURE----- --wac7ysb48OaltWcw-- From owner-freebsd-database@FreeBSD.ORG Wed Oct 12 21:34:53 2005 Return-Path: X-Original-To: freebsd-database@freebsd.org Delivered-To: freebsd-database@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BA04016A420 for ; Wed, 12 Oct 2005 21:34:53 +0000 (GMT) (envelope-from justin.bastedo@gmail.com) Received: from xproxy.gmail.com (xproxy.gmail.com [66.249.82.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id D397843D45 for ; Wed, 12 Oct 2005 21:34:50 +0000 (GMT) (envelope-from justin.bastedo@gmail.com) Received: by xproxy.gmail.com with SMTP id t12so138586wxc for ; Wed, 12 Oct 2005 14:34:50 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=IlhHSnYfcdEdvAnhJbxWSSxfzeygiu8ZxGjSRQbE6ZBRnZpkjQMJt7cVflAduShfXAeS+r1SBIDhYsmXwMlQsKZOrUf0EfSdGLdU6W5T1P12u12FBByOJEA26NcN/P9PUH+/EI+zcRA/p48qUjZ3tbAVDilScOtSi0E5xwHRpCg= Received: by 10.70.44.14 with SMTP id r14mr346198wxr; Wed, 12 Oct 2005 14:34:50 -0700 (PDT) Received: by 10.70.35.7 with HTTP; Wed, 12 Oct 2005 14:34:50 -0700 (PDT) Message-ID: <8a5255240510121434k6c2c1028x646120b245343a0f@mail.gmail.com> Date: Wed, 12 Oct 2005 14:34:50 -0700 From: Justin Bastedo To: NMH In-Reply-To: <20051012201622.49704.qmail@web32905.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <20051012201622.49704.qmail@web32905.mail.mud.yahoo.com> Cc: FreeBSD-DataBase , questions Subject: Re: Mysql server not able to stay running on anything but Linux? X-BeenThere: freebsd-database@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Database use and development under FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2005 21:34:53 -0000 Here at my work we are running a mysql 5.0 server on FreeBSD 5.4 using the AMD Dual Core 64 chips. current load is as follows: [MYSQL] Uptime: 1338832 Threads: 2 Questions: 15846788 Slow queries: 0 Opens: 0 Flush tables: 16 Open tables: 58 Queries per second avg: 11.836 [FREEBSD TOP] last pid: 29924; load averages: 0.00, 0.00, 0.00 up 35+01:59:20 16:2= 4:01 49 processes: 1 running, 48 sleeping CPU states: 0.5% user, 0.0% nice, 0.5% system, 0.0% interrupt, 99.0% id= le Mem: 156M Active, 1250M Inact, 176M Wired, 214M Buf, 2086M Free Swap: 2048M Total, 44K Used, 2048M Free Versions: >mysql -V mysql Ver 14.12 Distrib 5.0.12-beta, for portbld-freebsd5.4 (amd64) using = 4.3 >uname -a FreeBSD db 5.4-RELEASE-p7 FreeBSD 5.4-RELEASE-p7 #0: Wed Sep 7 14:12:34 CDT 2005 user@db:/usr/obj/usr/src/sys/BLITZ4SMP amd64 Our queries per second avg has been going up consistently and we have yet to have any problems. We haven't had any crashing though I can't quite say how the thread level is comparing to yours. I agree i would look into upgrading to 5.4-STABLE and see how that treats y= ou. As far as my experience the only real advantages I've found that linux has over FreeBSD is the type of file systems. Some of the file systems they support may give better performance to particular types of table types, but that is more if you have a stable db server that is hitting a bottle neck. -- Justin Bastedo At Gmail Dot Com From owner-freebsd-database@FreeBSD.ORG Wed Oct 12 21:41:56 2005 Return-Path: X-Original-To: freebsd-database@freebsd.org Delivered-To: freebsd-database@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E360916A420 for ; Wed, 12 Oct 2005 21:41:55 +0000 (GMT) (envelope-from drumslayer2@yahoo.com) Received: from web32913.mail.mud.yahoo.com (web32913.mail.mud.yahoo.com [68.142.206.60]) by mx1.FreeBSD.org (Postfix) with SMTP id 4A5DF43D45 for ; Wed, 12 Oct 2005 21:41:55 +0000 (GMT) (envelope-from drumslayer2@yahoo.com) Received: (qmail 17247 invoked by uid 60001); 12 Oct 2005 21:41:54 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=bekS5FkaepZrwpbM7mujU3H6RGOoqZuintRngeIHzhvJD8Gs34DJrgIrSpGTpw/QdJ56yvmyIVtYfwXspINnG9XcEdPDbrSW3Hb0zMof5bJngPuYUag7s2HuZs8cRREzVz6Szuju0qw9eM5RjAcD5VfAznhmGs18x+eRBPBRAOk= ; Message-ID: <20051012214154.17245.qmail@web32913.mail.mud.yahoo.com> Received: from [67.112.21.27] by web32913.mail.mud.yahoo.com via HTTP; Wed, 12 Oct 2005 14:41:54 PDT Date: Wed, 12 Oct 2005 14:41:54 -0700 (PDT) From: NMH To: Kris Kennaway In-Reply-To: <20051012210757.GA97184@xor.obsecurity.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: FreeBSD-DataBase , questions Subject: Re: Mysql server not able to stay running on anything but Linux? X-BeenThere: freebsd-database@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Database use and development under FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2005 21:41:56 -0000 --- Kris Kennaway wrote: > On Wed, Oct 12, 2005 at 01:16:22PM -0700, NMH wrote: > > Hi > > I am stuck with a delema and I feel like a damn > > troll. But.. I have a Mysql Database that I posted > > about earlier. It seems that it is only able to > not > > die by running on BSD 4.11 with Linux Threads. My > boss > > is convinced this means that Linux is better for > MySQL > > and wants that installed now. > > We even got a support contact from Mysql that so > far > > has gotten us nothing for almost a month while our > > production database server died up to 3 times a > day. > > (and lots of we're looking into it's) > > > > We were running fine but a little slow on FreeBSD > > 5.1-P11 and MySQL 4.0.18.(apperantly before a big > Lib > > change) We had to move quicker than we wanted to a > new > > server running FreeBSD 5.4 and MySQL 4.11 (becouse > of > > a dual HD death) Under production load the new 5.4 > > server fell over regulary. It has only now become > > stable by wiping it and running it on FreeBSD 4.11 > > with Linux Threads. (it regularly has over 400+ > > threads) > > Unfortunately you'll need to provide details of how > it "fell over" > (e.g. panic messages + backtraces). You might have > better luck with > 6.0-RELEASE or 5.4-STABLE, since a number of bugs > have been fixed in > 5.4-RELEASE..but without more details it's > impossible to say whether > that will help, or if you're encountering something > genuinely new. > > > Can anyone help or offer assistance to help > track > > this down? Perhaps also any annecdotes or examples > I > > can show my boss that other people have as busy > MYSQL > > databases on BSD 5.X. We paid 3K to Mysql for help > and > > so far they have been unable to offer any clues as > to > > why ours will not stay stable on anything but > Linux > > threads. I feel really sad that so far no one has > > responded to my posts and it feels like a victory > for > > linux. > > If I was your boss I'd be asking why mysql hasn't > delivered on their support contract. > > Kris Hi Kris Well I think support has many meanings. I decided to look at what he paid for and it says: We get "access" to the mysql devlopers... We get "access" to certified binaries. (none of which are FreeBSD) So, sad as it may seem, if your running on FreeBSD, $3000.00 buys you someone to talk to. It doesn't mean they have to say anything meaningful back or within any reasonable time. :( Also it doesn't cover tuning or code review etc. So if our problem is caused by anything like that.. they don't even have to tell us I would imagine. So.. that is why as always, these lists and fellow FreeBSD people are their own best support. Sadly this is good and bad. Without enough paying users, you don't tend to get as much attention. (this applies to hardware and software) NMH The Large Print Giveth And The Small Print Taketh Away -- Anon __________________________________ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs From owner-freebsd-database@FreeBSD.ORG Wed Oct 12 22:12:42 2005 Return-Path: X-Original-To: freebsd-database@freebsd.org Delivered-To: freebsd-database@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5E6BE16A443 for ; Wed, 12 Oct 2005 22:12:42 +0000 (GMT) (envelope-from drumslayer2@yahoo.com) Received: from web32913.mail.mud.yahoo.com (web32913.mail.mud.yahoo.com [68.142.206.60]) by mx1.FreeBSD.org (Postfix) with SMTP id A1C7C43D49 for ; Wed, 12 Oct 2005 22:12:41 +0000 (GMT) (envelope-from drumslayer2@yahoo.com) Received: (qmail 24238 invoked by uid 60001); 12 Oct 2005 22:12:41 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=MmVeEIfEG+z/7jGckhkhYvMJlkNogInInEyhl7tI41LGg+OL4Tj1puaoQ7ju5iMuYZCrrP4IFGHcC/Kw44vACiWuaegXXHJT8aLfLBBM/qsmb0puZRc7p8ub781meaft1FPOQVL0r9SwFe4k5/6tJhP5XvBlGmE1jLYofYlNo/g= ; Message-ID: <20051012221241.24236.qmail@web32913.mail.mud.yahoo.com> Received: from [67.112.21.27] by web32913.mail.mud.yahoo.com via HTTP; Wed, 12 Oct 2005 15:12:41 PDT Date: Wed, 12 Oct 2005 15:12:41 -0700 (PDT) From: NMH To: Justin Bastedo In-Reply-To: <8a5255240510121434k6c2c1028x646120b245343a0f@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: FreeBSD-DataBase , questions Subject: Re: Mysql server not able to stay running on anything but Linux? X-BeenThere: freebsd-database@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Database use and development under FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2005 22:12:42 -0000 --- Justin Bastedo wrote: > Here at my work we are running a mysql 5.0 server on > FreeBSD 5.4 using > the AMD Dual Core 64 chips. > > current load is as follows: > [MYSQL] > Uptime: 1338832 Threads: 2 Questions: 15846788 Slow > queries: 0 Opens: > 0 Flush tables: 16 Open tables: 58 Queries per > second avg: 11.836 > > [FREEBSD TOP] > last pid: 29924; load averages: 0.00, 0.00, 0.00 > up 35+01:59:20 16:24:01 > 49 processes: 1 running, 48 sleeping > CPU states: 0.5% user, 0.0% nice, 0.5% system, > 0.0% interrupt, 99.0% idle > Mem: 156M Active, 1250M Inact, 176M Wired, 214M Buf, > 2086M Free > Swap: 2048M Total, 44K Used, 2048M Free > > Versions: > >mysql -V > mysql Ver 14.12 Distrib 5.0.12-beta, for > portbld-freebsd5.4 (amd64) using 4.3 > > >uname -a > FreeBSD db 5.4-RELEASE-p7 FreeBSD 5.4-RELEASE-p7 #0: > Wed Sep 7 > 14:12:34 CDT 2005 > user@db:/usr/obj/usr/src/sys/BLITZ4SMP amd64 > > > Our queries per second avg has been going up > consistently and we have > yet to have any problems. We haven't had any > crashing though I can't > quite say how the thread level is comparing to > yours. > > I agree i would look into upgrading to 5.4-STABLE > and see how that treats you. > > As far as my experience the only real advantages > I've found that linux > has over FreeBSD is the type of file systems. Some > of the file systems > they support may give better performance to > particular types of table > types, but that is more if you have a stable db > server that is hitting > a bottle neck. > > -- > Justin Bastedo At Gmail Dot Com Hi Justin Right now here are our stats.. As you can see.. we beat it up pretty good. [MYSQL] Uptime: 174044 Threads: 396 Questions: 140154383 Slow queries: 16 Opens: 2622 Flush tables: 1 Open tables: 1189 Queries per second avg: 805.281 [FREEBSD TOP] (BSD 4.11-STABLE) last pid: 40914; load averages: 0.65, 0.69, 0.74 436 processes: 5 running, 431 sleeping CPU states: 0.5% user, 22.1% nice, 8.1% system, 0.4% interrupt, 69.4% idle Mem: 483M Active, 3046M Inact, 291M Wired, 136M Cache Buf, 6070K Free Swap: 8192M Total, 68K Used, 8192M Free NMH The Large Print Giveth And The Small Print Taketh Away -- Anon __________________________________ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs From owner-freebsd-database@FreeBSD.ORG Wed Oct 12 22:23:37 2005 Return-Path: X-Original-To: freebsd-database@freebsd.org Delivered-To: freebsd-database@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 453E916A41F for ; Wed, 12 Oct 2005 22:23:37 +0000 (GMT) (envelope-from justin.bastedo@gmail.com) Received: from xproxy.gmail.com (xproxy.gmail.com [66.249.82.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id C2DC843D4C for ; Wed, 12 Oct 2005 22:23:35 +0000 (GMT) (envelope-from justin.bastedo@gmail.com) Received: by xproxy.gmail.com with SMTP id t12so143784wxc for ; Wed, 12 Oct 2005 15:23:35 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=muzSjbSsjVVSu1wmrXXdHO3JOMpS54+Yg7lbpRsBkox7VZ6U0U8EVFrmJHi25q4yzAX3Csoxqlh44Gi3ViEXLOVoQlmi5f+xeMfOyRT7PHPsgrgAVhMwmeuAfSpO3cYTqRPR80wegxtCloZDJl2lvMDSRY+VgLk9+h7NokJH318= Received: by 10.70.131.10 with SMTP id e10mr354119wxd; Wed, 12 Oct 2005 15:23:35 -0700 (PDT) Received: by 10.70.35.7 with HTTP; Wed, 12 Oct 2005 15:23:35 -0700 (PDT) Message-ID: <8a5255240510121523p39c76508n99451832bbb9fdc9@mail.gmail.com> Date: Wed, 12 Oct 2005 15:23:35 -0700 From: Justin Bastedo To: NMH In-Reply-To: <20051012221241.24236.qmail@web32913.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <8a5255240510121434k6c2c1028x646120b245343a0f@mail.gmail.com> <20051012221241.24236.qmail@web32913.mail.mud.yahoo.com> Cc: FreeBSD-DataBase , questions Subject: Re: Mysql server not able to stay running on anything but Linux? X-BeenThere: freebsd-database@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Database use and development under FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2005 22:23:37 -0000 Yeah i still have quite a way to go to catch up on you on actual usage, but would be good to see if the problem persisted on the 5.4-STABLE release. As Kris said "a number of bugs have been fixed in 5.4-RELEASE.." it might be a good test. Also does this persist across all the versions of mysql? I know they just launched the RC of 5.0, may be able to leverage that a bit i know they are really pushing to get that released, and might be nice knowing that this problem is fixed in the new release. I'm definately interested to see what the final solution is on this matter. I can easily forsee our db load getting close to that level within the next couple of months depending on how these contracts play out. Justin From owner-freebsd-database@FreeBSD.ORG Wed Oct 12 22:55:45 2005 Return-Path: X-Original-To: freebsd-database@freebsd.org Delivered-To: freebsd-database@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E788716A41F for ; Wed, 12 Oct 2005 22:55:45 +0000 (GMT) (envelope-from drumslayer2@yahoo.com) Received: from web32901.mail.mud.yahoo.com (web32901.mail.mud.yahoo.com [68.142.206.48]) by mx1.FreeBSD.org (Postfix) with SMTP id 273F943D48 for ; Wed, 12 Oct 2005 22:55:45 +0000 (GMT) (envelope-from drumslayer2@yahoo.com) Received: (qmail 83675 invoked by uid 60001); 12 Oct 2005 22:55:44 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=2PqmFfYaF1s/NngGNVr9ryHnJRI6gk2XrV7qbf9y9u1QmvrPeXqt6Cq+5/QxFguafVSyWaemMYW7Dx0y9RU2S9O4p1ZtuAW91AANBJKjxOnZzvcnTEKBMTsXtoVL4VK89xA1OR0zvRfQgOyrFDnoqFDKlIOfvtDgdoKdsvzOixY= ; Message-ID: <20051012225544.83673.qmail@web32901.mail.mud.yahoo.com> Received: from [67.112.21.27] by web32901.mail.mud.yahoo.com via HTTP; Wed, 12 Oct 2005 15:55:44 PDT Date: Wed, 12 Oct 2005 15:55:44 -0700 (PDT) From: NMH To: justin.bastedo@gmail.com MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: FreeBSD-DataBase , questions Subject: Re: Mysql server not able to stay running on anything but Linux? X-BeenThere: freebsd-database@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Database use and development under FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2005 22:55:46 -0000 --- Justin Bastedo wrote: > Yeah i still have quite a way to go to catch up on > you on actual > usage, but would be good to see if the problem > persisted on the > 5.4-STABLE release. As Kris said "a number of bugs > have been fixed in > 5.4-RELEASE.." it might be a good test. Also does > this persist across > all the versions of mysql? I know they just launched > the RC of 5.0, > may be able to leverage that a bit i know they are > really pushing to > get that released, and might be nice knowing that > this problem is > fixed in the new release. Hi again :) Well we built 3 fairly identical servers for testing and to try replication. At first they all had BSD 5.4-RELEASE. One had -P4 the other 2 had -p6 So they were pretty high up there on the patch levels. Also to stop them from falling over constantly we had to insert: (as it seemed the other threads could not support it at all!) /etc/libmap.conf libpthread.so libthr.so libpthread.so.1 libthr.so.1 Also no one from Mysql seemed to think going to STABLE would do anything. It's a very very tough thing to play lets see what happens if.. On production servers. The time it takes us just to bounce from one machine to another is pretty decent. Let alone downtime etc. Hence my boss's push for Linux. "Since MYSQL says it will definatly work on Linux.. We should switch to Linux." > I'm definately interested to see what the final > solution is on this > matter. I can easily forsee our db load getting > close to that level > within the next couple of months depending on how > these contracts play > out. Well, I will post whatever happens. As I said the most annoying part to all this is that so far we have been unable to crash the 5.4 servers via any testing means we (and Mysql) could come up with. Yet in production they fell over as soon as the load started to climb and points in between. But never on the slope of decreasing load for the day. (always felt that was odd) IE it would die on the bell curve increases but never at the coresponding downward cycle of the daily busy curve. If anyone knows of any particulary evil testing programs I would love to know. It's tough to simulate 30+ servers asking lots questions and forcing tons of writes. Thanks! NMH Nicole The Large Print Giveth And The Small Print Taketh Away -- Anon __________________________________ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com From owner-freebsd-database@FreeBSD.ORG Thu Oct 13 02:56:38 2005 Return-Path: X-Original-To: freebsd-database@freebsd.org Delivered-To: freebsd-database@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 02B6A16A41F; Thu, 13 Oct 2005 02:56:38 +0000 (GMT) (envelope-from grog@lemis.com) Received: from ext-gw.lemis.com (ext-gw.lemis.com [150.101.14.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 094AE43D53; Thu, 13 Oct 2005 02:56:33 +0000 (GMT) (envelope-from grog@lemis.com) Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.135]) by ext-gw.lemis.com (Postfix) with ESMTP id AD55F131BB7; Thu, 13 Oct 2005 12:26:32 +0930 (CST) Received: by wantadilla.lemis.com (Postfix, from userid 1004) id 81F1D84F7C; Thu, 13 Oct 2005 12:26:32 +0930 (CST) Date: Thu, 13 Oct 2005 12:26:32 +0930 From: Greg 'groggy' Lehey To: NMH , Kris Kennaway , justin.bastedo@gmail.com Message-ID: <20051013025632.GN49168@wantadilla.lemis.com> References: <20051012225544.83673.qmail@web32901.mail.mud.yahoo.com> <20051012210757.GA97184@xor.obsecurity.org> <20051012214154.17245.qmail@web32913.mail.mud.yahoo.com> <20051012201622.49704.qmail@web32905.mail.mud.yahoo.com> <20051012210757.GA97184@xor.obsecurity.org> <20051012201622.49704.qmail@web32905.mail.mud.yahoo.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="GTjM2GI4zMe3xHZ5" Content-Disposition: inline In-Reply-To: <20051012225544.83673.qmail@web32901.mail.mud.yahoo.com> <20051012214154.17245.qmail@web32913.mail.mud.yahoo.com> <20051012210757.GA97184@xor.obsecurity.org> <20051012201622.49704.qmail@web32905.mail.mud.yahoo.com> User-Agent: Mutt/1.4.2.1i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 Cc: FreeBSD-DataBase , questions Subject: Re: Mysql server not able to stay running on anything but Linux? X-BeenThere: freebsd-database@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Database use and development under FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2005 02:56:38 -0000 --GTjM2GI4zMe3xHZ5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Note the From: address. On Wednesday, 12 October 2005 at 13:16:22 -0700, NMH wrote: > > I am stuck with a delema and I feel like a damn troll. But.. I have > a Mysql Database that I posted about earlier. It seems that it is > only able to not die by running on BSD 4.11 with Linux Threads. My > boss is convinced this means that Linux is better for MySQL and > wants that installed now. > > We even got a support contact from Mysql that so far has gotten us > nothing for almost a month while our production database server died > up to 3 times a day. (and lots of we're looking into it's) One of the reasons why you haven't got much more than "we're looking into it"s is because we haven't been able to reproduce the problem; you acknowledge this in follow-up mail quoted below. As you know from various threads on the FreeBSD lists, including this one, the typical answer is "works fine for me". That doesn't mean that we're not taking your problems seriously, but we do have a significant issue just reproducing the problem. We have a number of choices: 1. Try different hardware or a different version of FreeBSD. It's conceivable that there's something about your specific hardware, or about the combination of i386 kernel on amd64 in general, that triggers the problem. 2. Do debugging on your production servers. This isn't really a choice at all: it would involve even more down time. 3. Get you to run a more stable version of FreeBSD while we investigate the problem. This is the method we chose. I haven't heard from you since the weekend, so I hope I'm correct in understanding that you currently don't have stability problems. On our side, we have installed FreeBSD 5.4 on one of our internal machines, and we're trying to reproduce the problem there. > We were running fine but a little slow on FreeBSD 5.1-P11 and MySQL > 4.0.18.(apperantly before a big Lib change) We had to move quicker > than we wanted to a new server running FreeBSD 5.4 and MySQL 4.11 > (becouse of a dual HD death) Under production load the new 5.4 > server fell over regulary. It has only now become stable by wiping > it and running it on FreeBSD 4.11 with Linux Threads. (it regularly > has over 400+ threads) Kris obviously understood that by this statement you meant a kernel crash. My understanding is that only the mysqld server is crashing. Is this still correct? > I want to try FreeBSD 5.4 AMD64 (the machines are Opteron) or 6.0 > but my boss feels that would be a waste given that MYSQL doesn't > support Mysql on AMD64 well enough. I think it would be a good idea to try this. It's one of the things that we intend to do in-house as soon as we can reproduce the problem at all. > Can anyone help or offer assistance to help track this down? Perhaps > also any annecdotes or examples I can show my boss that other people > have as busy MYSQL databases on BSD 5.X. We paid 3K to Mysql for > help and so far they have been unable to offer any clues as to why > ours will not stay stable on anything but Linux threads. Have you had any kind of crash under 4.x? I don't think that the issue is so much linuxthreads as 5.x. > As I say I only manage the server, I don't program the databases. Is > there anything I should/could look for database wise that could > trigger such things? So far we've had the machine up in-house and have not reproduced the problem. If you have a spare machine that we could run under more typical conditions on your premises, this might help. On Wednesday, 12 October 2005 at 17:07:57 -0400, Kris Kennaway wrote: > > Unfortunately you'll need to provide details of how it "fell over" > (e.g. panic messages + backtraces). As I mention above, I think this is only a server crash. I mentioned this on the list a couple of weeks ago: all the backtraces I have seen have been a SIGSEGV out of mutex_unlock_common. >> Can anyone help or offer assistance to help track this down? >> Perhaps also any annecdotes or examples I can show my boss that >> other people have as busy MYSQL databases on BSD 5.X. We paid 3K to >> Mysql for help and so far they have been unable to offer any clues >> as to why ours will not stay stable on anything but Linux >> threads. I feel really sad that so far no one has responded to my >> posts and it feels like a victory for linux. > > If I was your boss I'd be asking why mysql hasn't delivered on their > support contract. Indeed. I think we have, though. There's a certain class of bugs which are almost impossible to fix because they're so hard to chase down. This is one of them. On Wednesday, 12 October 2005 at 14:41:54 -0700, NMH wrote: > --- Kris Kennaway wrote: > >> If I was your boss I'd be asking why mysql hasn't delivered on >> their support contract. > > Well I think support has many meanings. I decided to look at what he > paid for and it says: > We get "access" to the mysql devlopers... > We get "access" to certified binaries. (none of which > are FreeBSD) > > So, sad as it may seem, if your running on FreeBSD, $3000.00 buys > you someone to talk to. It doesn't mean they have to say anything > meaningful back or within any reasonable time. :( We try to handle all problems within a reasonable time. The fact that you're running FreeBSD does mean that you don't get certified binaries, but that's the only drawback. And the fact that the time for this problem has been unreasonable has nothing to do with the fact you're running FreeBSD: it's because it's a bugger to track down. > Also it doesn't cover tuning or code review etc. So if our problem > is caused by anything like that.. they don't even have to tell us I > would imagine. FWIW, my personal opinion (I don't speak for MySQL, but I don't think the official line would be much different) is that a database server shouldn't crash. If it does, it's a bug. The question we need to clarify is not whether your code is a problem, but whether the crash is due to FreeBSD or MySQL. Currently the evidence is pointing to FreeBSD, but we're keeping an open mind on that. > So.. that is why as always, these lists and fellow FreeBSD people > are their own best support. Apparently we've given you the impression that we're not doing anything. I'm sorry about that, and it's certainly not the case. So far all these lists have done is to provide the valuable information that this is not a general problem. On Wednesday, 12 October 2005 at 15:55:44 -0700, NMH wrote: > --- Justin Bastedo wrote: > > > Also no one from Mysql seemed to think going to STABLE would do > anything. It's a very very tough thing to play lets see what > happens if.. On production servers. The time it takes us just to > bounce from one machine to another is pretty decent. Let alone > downtime etc. This is puzzling. The bug report is visible to the world at http://bugs.mysql.com/bug.php?id=12251 : it was entered against 5.4-STABLE. I did express the opinion, based on code analysis, that upgrading wouldn't make much difference. >> I'm definately interested to see what the final solution is on this >> matter. I can easily forsee our db load getting close to that level >> within the next couple of months depending on how these contracts >> play out. Whether or not you have a contract, I'd be very interested in hearing of any problems you have with high load. Obviously you'll get better support if you do have a contract, Nicole's impressions notwithstanding. > Well, I will post whatever happens. As I said the most annoying > part to all this is that so far we have been unable to crash the > 5.4 servers via any testing means we (and Mysql) could come up > with. Yet in production they fell over as soon as the load started > to climb and points in between. But never on the slope of > decreasing load for the day. (always felt that was odd) IE it would > die on the bell curve increases but never at the coresponding > downward cycle of the daily busy curve. I noticed that too; I suspect it's not statistically relevant. > If anyone knows of any particulary evil testing programs I would > love to know. It's tough to simulate 30+ servers asking lots > questions and forcing tons of writes. If you (anyone) do have useful input, please keep me in the loop too. Greg -- Greg Lehey, Senior Software Engineer MySQL AB, http://www.mysql.com/ Echunga, South Australia See complete headers for address and phone numbers. Are you MySQL certified? http://www.mysql.com/certification/ --GTjM2GI4zMe3xHZ5 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (FreeBSD) iD8DBQFDTczgIubykFB6QiMRAs+yAJ9YsDJE1iooFZzplYwcmm9KFJC7JwCcCtMy FfcmSvl0PU3T9UD0EWgm5mY= =PmgN -----END PGP SIGNATURE----- --GTjM2GI4zMe3xHZ5-- From owner-freebsd-database@FreeBSD.ORG Thu Oct 13 04:31:17 2005 Return-Path: X-Original-To: freebsd-database@freebsd.org Delivered-To: freebsd-database@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5D88F16A41F for ; Thu, 13 Oct 2005 04:31:17 +0000 (GMT) (envelope-from drumslayer2@yahoo.com) Received: from web32913.mail.mud.yahoo.com (web32913.mail.mud.yahoo.com [68.142.206.60]) by mx1.FreeBSD.org (Postfix) with SMTP id 2FAFD43D49 for ; Thu, 13 Oct 2005 04:31:16 +0000 (GMT) (envelope-from drumslayer2@yahoo.com) Received: (qmail 4695 invoked by uid 60001); 13 Oct 2005 04:31:15 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=wjYzSKR6jc0FZFwAzG9n+22uIti7nuwrpwCnB2XrZhMGl2aeO8YD+oLC0bgqZKzxYBOUpgdouu3hL3g+6gJEXnJL0HTgq5clbgxJ0XYYwosu/Qi+sWpQQtjoy2i7HyHDjgYdF+k9o3/HNpoJALfr2x//4msd5I2bxNXfdMjNuyA= ; Message-ID: <20051013043115.4693.qmail@web32913.mail.mud.yahoo.com> Received: from [67.112.21.27] by web32913.mail.mud.yahoo.com via HTTP; Wed, 12 Oct 2005 21:31:15 PDT Date: Wed, 12 Oct 2005 21:31:15 -0700 (PDT) From: NMH To: Greg 'groggy' Lehey , Kris Kennaway , justin.bastedo@gmail.com In-Reply-To: <20051013025632.GN49168@wantadilla.lemis.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: FreeBSD-DataBase , questions Subject: Re: Mysql server not able to stay running on anything but Linux? X-BeenThere: freebsd-database@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Database use and development under FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2005 04:31:17 -0000 --- Greg 'groggy' Lehey wrote: > Note the From: address. > > On Wednesday, 12 October 2005 at 13:16:22 -0700, NMH > wrote: > > > > I am stuck with a delema and I feel like a damn > troll. But.. I have > > a Mysql Database that I posted about earlier. It > seems that it is > > only able to not die by running on BSD 4.11 with > Linux Threads. My > > boss is convinced this means that Linux is better > for MySQL and > > wants that installed now. > > > > We even got a support contact from Mysql that so > far has gotten us > > nothing for almost a month while our production > database server died > > up to 3 times a day. (and lots of we're looking > into it's) > > One of the reasons why you haven't got much more > than "we're looking > into it"s is because we haven't been able to > reproduce the problem; > you acknowledge this in follow-up mail quoted below. > > As you know from various threads on the FreeBSD > lists, including this > one, the typical answer is "works fine for me". > That doesn't mean > that we're not taking your problems seriously, but > we do have a > significant issue just reproducing the problem. We > have a number of > choices: > > 1. Try different hardware or a different version of > FreeBSD. It's > conceivable that there's something about your > specific hardware, > or about the combination of i386 kernel on amd64 > in general, that > triggers the problem. Yes possible. However the same hardware was used for the earlier version machine that worked fine. IE Freebsd 5.1-RELEASE-P11 and Mysql 4.0.18 worked on the same hardware. > 2. Do debugging on your production servers. This > isn't really a > choice at all: it would involve even more down > time. Yea not really an option. > 3. Get you to run a more stable version of FreeBSD > while we > investigate the problem. This is the method we > chose. I haven't > heard from you since the weekend, so I hope I'm > correct in > understanding that you currently don't have > stability problems. > On our side, we have installed FreeBSD 5.4 on > one of our internal > machines, and we're trying to reproduce the > problem there. So far we have had only one crash that seemed to have been SCSI related. So far it has not happend again. One problem with replication that was a coding issue. > > We were running fine but a little slow on FreeBSD > 5.1-P11 and MySQL > > 4.0.18.(apperantly before a big Lib change) We had > to move quicker > > than we wanted to a new server running FreeBSD 5.4 > and MySQL 4.11 > > (becouse of a dual HD death) Under production load > the new 5.4 > > server fell over regulary. It has only now become > stable by wiping > > it and running it on FreeBSD 4.11 with Linux > Threads. (it regularly > > has over 400+ threads) > > Kris obviously understood that by this statement you > meant a kernel > crash. My understanding is that only the mysqld > server is crashing. > Is this still correct? Yes only Mysqld would crash. Sometimes brb and autorestart with minimal damage. Other times it would die a horrible death and damage tons of data on its way out. > > I want to try FreeBSD 5.4 AMD64 (the machines are > Opteron) or 6.0 > > but my boss feels that would be a waste given that > MYSQL doesn't > > support Mysql on AMD64 well enough. > > I think it would be a good idea to try this. It's > one of the things > that we intend to do in-house as soon as we can > reproduce the problem > at all. Yes however as I pointed out.. Just Trying things on a production database is not desirable without some serious indicators that its worthwhile. Last you wrote you said you doubted that would do anything I believe. Also support for amd64 based mysql is listed as Limited. As you said reproducing the problem is the key. However as I suggested I would have thought that if mysql were really into solving the problem, someone would have requested a login on the box to look at our queries to see how they are. Are they 60% reads 40% writes, are they many divergent queries bundled together.. etc. IE come and see our production database in action to see what needs to be replicated. I haven't seen anything like this. Now I don't know much but to me if I can't replicate something it's becouse I don't know enough about it. > > Can anyone help or offer assistance to help track > this down? Perhaps > > also any annecdotes or examples I can show my boss > that other people > > have as busy MYSQL databases on BSD 5.X. We paid > 3K to Mysql for > > help and so far they have been unable to offer any > clues as to why > > ours will not stay stable on anything but Linux > threads. > > Have you had any kind of crash under 4.x? I don't > think that the > issue is so much linuxthreads as 5.x. Well. 5.1.. Old lib style No crashes 5.4 .. Crashes like mad 5.4 with libmap.conf to other threads works better.. 4.11 with Linux threads.. No Crashes. So yea.. But thats the other issue that makes it hard. Is it Mysql or is it Freebsd or the interaction and memory sharing going on. I hope that someone here who deals with these things might actually read these emails. Its just very frustrating. > > As I say I only manage the server, I don't program > the databases. Is > > there anything I should/could look for database > wise that could > > trigger such things? > > So far we've had the machine up in-house and have > not reproduced the > problem. If you have a spare machine that we could > run under more > typical conditions on your premises, this might > help. We have one machine still running 5.4. But what to do with it? I would love to convert it to amd64 but we need some machine to still 5.4 so we can have something to test. > On Wednesday, 12 October 2005 at 17:07:57 -0400, > Kris Kennaway wrote: > > > > Unfortunately you'll need to provide details of > how it "fell over" > > (e.g. panic messages + backtraces). I didn't see this email. We did supply all that to you (greg and Mysql) as far as I know. If you could share what is needed that would be appreciatted. > As I mention above, I think this is only a server > crash. I mentioned > this on the list a couple of weeks ago: all the > backtraces I have seen > have been a SIGSEGV out of mutex_unlock_common. Your opinion on this given what you have seen of our issue? > >> Can anyone help or offer assistance to help track > this down? > >> Perhaps also any annecdotes or examples I can > show my boss that > >> other people have as busy MYSQL databases on BSD > 5.X. We paid 3K to > >> Mysql for help and so far they have been unable > to offer any clues > >> as to why ours will not stay stable on anything > but Linux > >> threads. I feel really sad that so far no one has > responded to my > >> posts and it feels like a victory for linux. > > > > If I was your boss I'd be asking why mysql hasn't > delivered on their > > support contract. > > Indeed. I think we have, though. There's a certain > class of bugs > which are almost impossible to fix because they're > so hard to chase > down. This is one of them. Lucky us. :( But does that also mean that anyone else running 5.4 and Mysql will likely run into this when they reach our level of activity? > On Wednesday, 12 October 2005 at 14:41:54 -0700, NMH > wrote: > > --- Kris Kennaway wrote: > > > >> If I was your boss I'd be asking why mysql hasn't > delivered on > >> their support contract. > > > > > Well I think support has many meanings. I decided > to look at what he > > paid for and it says: > > We get "access" to the mysql devlopers... > > We get "access" to certified binaries. (none of > which > > are FreeBSD) > > > > So, sad as it may seem, if your running on > FreeBSD, $3000.00 buys > > you someone to talk to. It doesn't mean they have > to say anything > > meaningful back or within any reasonable time. :( > > We try to handle all problems within a reasonable > time. The fact that > you're running FreeBSD does mean that you don't get > certified > binaries, but that's the only drawback. And the > fact that the time > for this problem has been unreasonable has nothing > to do with the fact > you're running FreeBSD: it's because it's a bugger > to track down. Right. Which is why I am here. Hoping to widing the audience and ask for help rather than having to switch to Linux to get this solved. I didn't see you posting anything to ask others so I thought I would. As I said sometimes the gift and curse of FreeBSD is most of its users not needing or relying on commercial support for hardware and software things. So we can be overlooked as a market. Greg. if you wouldn't mind sharing the info with Kris I would appreciate it. If there is anything else needed just ask. Be well! === message truncated === NMH The Large Print Giveth And The Small Print Taketh Away -- Anon __________________________________ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com