From owner-freebsd-arch@FreeBSD.ORG Fri Jul 25 07:17:47 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 60D3C43D; Fri, 25 Jul 2014 07:17:47 +0000 (UTC) Received: from cyrus.watson.org (cyrus.watson.org [198.74.231.69]) by mx1.freebsd.org (Postfix) with ESMTP id 35A092D81; Fri, 25 Jul 2014 07:17:47 +0000 (UTC) Received: from [10.0.1.9] (host86-132-108-36.range86-132.btcentralplus.com [86.132.108.36]) by cyrus.watson.org (Postfix) with ESMTPSA id 38BAC46B43; Fri, 25 Jul 2014 03:17:43 -0400 (EDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: [RFC] ASLR Whitepaper and Candidate Final Patch From: "Robert N. M. Watson" In-Reply-To: <20140724175704.GT29618@pwnie.vrt.sourcefire.com> Date: Fri, 25 Jul 2014 08:17:36 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <96C72773-3239-427E-A90B-D05FF0F5B782@freebsd.org> <20140720201858.GB29618@pwnie.vrt.sourcefire.com> <20140723004543.GH29618@pwnie.vrt.sourcefire.com> <20140724175704.GT29618@pwnie.vrt.sourcefire.com> To: Shawn Webb X-Mailer: Apple Mail (2.1878.6) Cc: PaX Team , Pedro Giffuni , Oliver Pinter , Bryan Drewery , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2014 07:17:47 -0000 On 24 Jul 2014, at 18:57, Shawn Webb wrote: >>> I think someone has already commented that Peter Holm's help might = be=20 >>> enlisted; you have have seen his 'stress2' suite, which could help = with=20 >>> stability testing. >>=20 >> I'll take a look at that, too. Thanks a lot for your suggestions and >> feedback. >=20 > The unixbench results are in. The overall scores are below. >=20 > ASLR Disabled: 456.33 > ASLR Enabled: 357.05 > No ASLR: 474.03 >=20 > I've uploaded the raw results to > http://0xfeedface.org/~shawn/aslr/2014-07-24_benchmark.tar.gz >=20 > Take these results with a grain of salt, given that some of = unixbench's > test are filesystem-related and I'm running ZFS on an old laptop with > little RAM. It does show that there is a performance impact when ASLR = is > enabled. Just in case you've not spotted it, there's some useful benchmarking = advice here: https://wiki.freebsd.org/BenchmarkAdvice Unfortunately, the numbers above are a bit opaque, as it's not clear = whether the differences/non-differences are statistically significant. = Likewise, we'd expect that ASLR might impact some types of behaviour = more than others, and so reduction to a single number can overlook = problems or overemphasise differences. For now, the key thing is really = that there not be any measurable performance difference when ASLR is = disabled, and the numbers above make it a bit unclear if that is the = case. The numbers are definitely difference above -- but perhaps this is = a result of non-essential code generation differences, noise in the run, = etc. Typically, you would want to use a technique such as a t-test to = compare runs and decide if the difference is significant. Tools such as = ministat are very useful here, although you have to be a bit careful as = most performance measurements are already arithmetic means due to the = need to run individual instances of the operation of interest many = times, and comparing means of means is a messy business. The next direction will be to dig more into areas where there are = statistically significant changes to decide whether they are caused by = ASLR, or perhaps are just non-essential differences in code generation. = It may be useful to consider using a suite like 'libmicro' that can = drill into individual system-call behaviour more, as well as = larger-scale benchmarks that consider the behaviour of applications with = realisticish workloads -- Postgres has been of particular interest = lately. Robert=