From owner-freebsd-stable@FreeBSD.ORG Fri Jun 17 14:38:24 2005 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2033616A41C for ; Fri, 17 Jun 2005 14:38:24 +0000 (GMT) (envelope-from dsze@alumni.uwaterloo.ca) Received: from mail.distrust.net (mail.distrust.net [69.93.230.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id ECAAC43D1D for ; Fri, 17 Jun 2005 14:38:23 +0000 (GMT) (envelope-from dsze@alumni.uwaterloo.ca) Received: from eeyore.distrust.net (CPE00a0c978120d-CM00122570472e.cpe.net.cable.rogers.com [70.28.248.121]) (authenticated bits=0) by mail.distrust.net (8.13.3/8.13.3) with ESMTP id j5HEcMmb094081 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Fri, 17 Jun 2005 09:38:22 -0500 (CDT) (envelope-from dsze@alumni.uwaterloo.ca) Message-Id: <6.2.1.2.2.20050617103807.058c6fa8@mail.distrust.net> X-Mailer: QUALCOMM Windows Eudora Version 6.2.1.2 Date: Fri, 17 Jun 2005 10:38:34 -0400 To: freebsd-stable@freebsd.org From: David Sze Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: ClamAV 0.85.1/944/Thu Jun 16 16:33:33 2005 on mail.distrust.net X-Virus-Status: Clean Subject: Re: FreeBSD MySQL still WAY slower than Linux X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jun 2005 14:38:24 -0000 At 05:15 PM 16/06/2005 +0100, Steve Roome wrote this to All: >Thank you all for your suggestions on this thread, here's a brief >breakdown of most of the ideas from people: > >Billy Newsom: COMPILER, DISK, MYSQLVERSION >Daniel Eischen: +/-HTT, Thread scopes >Greg Lehey: MALLOC >Guy Helmer: PREEMPTIVE, vfs.read_max >Jon Dama: David Xu's Thrds, Ptmalloc, cpu affinity, sched+hwcacheing >Kris Kennaway: +/-HTT >Robert Watson: Thread scopes, LIBTHR/Linuxthr on 5 and 6?, LOCKING, > HTT, SMP/UP >Thomas Hurst: FreeBSD-current, Don't overload mysql! >Vladimir Chukharev: COMPILEOPTS, TABLETYPES >Xin Li: PROFILE, HTT insignificant > >The bad news is that I've not managed to get very far at all lately as >MySQL has been crashing too much to even stop and test stuff >elsewhere. > >The good news though, is that the Mysql folks have agreed to setup >tests to profile mysql on identical hardware running FreeBSD and Linux >with an aim to find out exactly where the problem really is. They >reckon they'll spend at least two weeks trying to find out why Linux >is so much faster - if they do this right I'd be surprised if we can't >improve things quite a lot. > >Also, it'd be good if any of you who still have an interest in this >could add any ideas or suggestions that may help *them* with their >testing. If so, just get in touch with me asap before they get too >stuck into anything that might prove fruitless. > >Here's hoping we can get MySQL running as well on FreeBSD as it ought >to. I just spent a couple days comparing MySQL 4.1 performance on: FreeBSD/amd64 5.4-RELEASE (libpthread, system and process scope) FreeBSD/amd64 6.0-CURRENT (libpthread, libthr, system and process scope) CentOS/amd64 4.0 (i.e. RHEL4.0) I couldn't get libthr to work on FreeBSD/amd64 5.4-RELEASE, mysqld would immediately coredump and the stacktrace looked like it was corrupted (i.e. hundreds of stack frames, all of which were ???). This was the hardware: Dell PowerEdge 2850 Dual Xeon 3.6GHz EMT64, HTT disabled 4GB RAM amr RAID controller w/256MB battery backed cache 5 x 73GB 15K RPM SCSI disks configured as one RAID5 w/64KB stripe I was seeing the same sort of super-smack numbers that everyone's been reporting, i.e. that Linux box was twice as fast as FreeBSD. It turns out that the problem was the same thing everyone usually points the finger at, but no one actually mentioned this time: Linux mounts its partitions async by default. I don't have the exact numbers in front of me right now, but these were the ballpark figures (I'm not going to separate out results for all of the different threading libraries for FreeBSD because the deltas weren't huge): super-smack select-key 5.4-RELEASE ~20,000 queries/second 6.0-CURRENT ~24,000 queries/second CentOS w/async ~36,000 queries/second CentOS w/sync ~26,000 queries/second super-smack update-select 5.4-RELEASE ~4,000 queries/second 6.0-CURRENT ~4,500 queries/second CentOS w/async ~7,500 queries/second CentOS w/sync ~750 queries/second That last CentOS number is not a typo, it was an order of magnitude slower. I didn't try other file systems on CentOS, just the default ext3. It's possible that reiserfs or xfs might not be as affected by switching from async to sync. So my production server is now happily running mysql 4.1 on 6.0-CURRENT :).