From owner-freebsd-performance@FreeBSD.ORG Thu Jun 13 12:02:38 2013 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 295BB548 for ; Thu, 13 Jun 2013 12:02:38 +0000 (UTC) (envelope-from remy@activnetworks.com) Received: from fr-exchange.activnetworks.com (anwadmin.net8.nerim.net [213.41.185.85]) by mx1.freebsd.org (Postfix) with ESMTP id ACB0E192B for ; Thu, 13 Jun 2013 12:02:36 +0000 (UTC) Received: from rn.activnetworks.com ([192.168.1.100]) by fr-exchange.activnetworks.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 13 Jun 2013 14:01:27 +0200 Message-ID: <51B9B497.70800@activnetworks.com> Date: Thu, 13 Jun 2013 14:01:27 +0200 From: Remy Nonnenmacher User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130522 Thunderbird/17.0.6 MIME-Version: 1.0 To: Mark Felder Subject: Re: Scaling and performance issues with FreeBSD 9 (& 10) on 4 socket systems References: <20130612225849.GA2858@dragon.NUXI.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 13 Jun 2013 12:01:27.0579 (UTC) FILETIME=[BBE99EB0:01CE682D] Cc: "freebsd-performance@freebsd.org" X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Jun 2013 12:02:38 -0000 On 06/13/13 13:32, Mark Felder wrote: > On Wed, 12 Jun 2013 17:58:49 -0500, David O'Brien > wrote: > >> We found FreeBSD 8.4 to perform better than FreeBSD 9.1, and Linux >> considerably better than both on the same machine. > > http://svnweb.freebsd.org/base?view=revision&revision=241246 > > The above link is likely why 8.4 is better than 9.1 on the same machine. > >> We've tried various things and haven't been able to explain why FreeBSD >> isn't scaling on the new hardware. Nor why it performs so much worse >> than FreeBSD on the older "M2" machines. > > The CPUs between those machines are quite different. I'm sure we're > looking at different cache sizes, different behavior for the > hyperthreading, etc. I'm sure others would be greatly interested in you > providing the same benchmark results for a recent snapshot of HEAD as well. > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" We had same problem on 4x12 cores (AMD) machines. After investigating using hwpmc, it appears that performance was killed by a scheduler function trying to find "least used cpu" that unfortunately works on contended structures (ie: lots a cores are fighting to get works). A solution was found by using artificially long queue of stuck process (steal_thresh bumped to over 8) and by cpu affinity crafting. Was a year ago and from my memory. I guess you may give a try to see if it helps. Disregard is a scheduler specialist contradicts. Thanks.