From owner-freebsd-questions@FreeBSD.ORG Tue May 18 16:45:11 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A0D5106567C for ; Tue, 18 May 2010 16:45:11 +0000 (UTC) (envelope-from fireduck@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9F9598FC25 for ; Tue, 18 May 2010 16:45:10 +0000 (UTC) Received: by fxm19 with SMTP id 19so2593225fxm.13 for ; Tue, 18 May 2010 09:45:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=25Q8VicBx72aaRVUA8rPHduYw62BMa4qRvDy1/K75+I=; b=orU6ixOoEntF9reJ61RORvsJfg/XFUo+XByS6Z6Gwi093zgaXTY5zWn/LfLyVDrnXY TFL3pIFseIIJn27MUb+LQw8o/ANs4P39nKaSrqnOvfVsU6F0+o7E6WBy+WTu5/nD8/Q2 L6nnWmxX9tooKwcejmWrOG9G6pjfV1PbsUKk8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=k2uSQVY6BgZIrIWv3YpLzeHyYGj+ZYLYMA7b3RtiU6fqz6v+rUEoIXRrMXGi5A3HuD eQdGEAVQIoi58u0g48m635r89YOgbnYtziOszCS7/APItBWDyLFSKcln5qKFFheLjWj/ Qb6IFBUMH/RcfLRzOUrFqAqkTL1OSkJ4qje5U= MIME-Version: 1.0 Received: by 10.223.16.84 with SMTP id n20mr8431949faa.94.1274201109215; Tue, 18 May 2010 09:45:09 -0700 (PDT) Received: by 10.223.116.193 with HTTP; Tue, 18 May 2010 09:45:09 -0700 (PDT) In-Reply-To: References: Date: Tue, 18 May 2010 09:45:09 -0700 Message-ID: From: Joseph Gleason To: freebsd-questions@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: Where has my gbde write performance gone? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 May 2010 16:45:11 -0000 For whatever it is worth, if I use geli rather than gbde I get normal (~30MB/s) performance. I also get the same slow gbde performance on 8.1-PRERELEASE as of last nigh= t. I've make a kernel swaping in files from 7.2 source to see if I got any improvement. I pulled in: geom_dev.c (with some hacks to get it compile) geom_slice.c geom_io.c None of those improved performance. If anyone has any suggestions for things to try, let me know. I am fine with switching to geli for some applications but I have about 12TB under gbde. That would be somewhat of a bear to copy over into geli. On Mon, May 17, 2010 at 7:32 PM, Joseph Gleason wrote: > Sometime between FreeBSD 7.2-RELEASE-p4 and 8.0-RELEASE write > performance of gbde encrypted devices seems to have dropped > significantly. =A0A system I have running 7.2 seems to run gbde drives > at or near the drive max rate (30-40MB/s) while I am seeing less than > 10% of that on 8.0 systems. > > I get the same slow writes on 8.0-RELEASE-p2 as well as 8.0-RELEASE. > > Here is an example on a fresh 8.0 install which shows gbde taking the > drive write performance of 40 MB/s down to 2.6 MB/s: > > lab# uname -a > FreeBSD lab.int.fireduck.com 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Sat > Nov 21 15:02:08 UTC 2009 > root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC =A0amd64 > > lab# dd if=3D/dev/urandom of=3D/dev/ad4s1d bs=3D32k count=3D32k > 32768+0 records in > 32768+0 records out > 1073741824 bytes transferred in 25.130537 secs (42726577 bytes/sec) > > lab# gbde init /dev/ad4s1d > Enter new passphrase: > Reenter new passphrase: > > lab# gbde attach /dev/ad4s1d > Enter passphrase: > > lab# dd if=3D/dev/urandom of=3D/dev/ad4s1d.bde bs=3D32k count=3D32k > 32768+0 records in > 32768+0 records out > 1073741824 bytes transferred in 401.097004 secs (2677013 bytes/sec) > > iostat from while that last 'dd' was running: > > =A0 =A0 =A0 tty =A0 =A0 =A0 =A0 =A0 =A0 ad4 =A0 =A0 =A0 =A0 =A0 =A0 cpu > =A0tin =A0tout =A0KB/t tps =A0MB/s =A0us ni sy in id > =A0 0 =A0 =A022 =A05.67 483 =A02.67 =A0 0 =A00 =A04 =A01 96 > =A0 0 =A0 =A066 =A05.67 509 =A02.82 =A0 0 =A00 =A04 =A01 95 > =A0 0 =A0 =A022 =A05.69 514 =A02.86 =A0 0 =A00 =A06 =A01 94 > =A0 0 =A0 =A022 =A05.67 506 =A02.80 =A0 0 =A00 =A06 =A01 93 > =A0 0 =A0 =A022 =A05.67 472 =A02.61 =A0 0 =A00 =A04 =A01 95 > > > iostat on a FreeBSD 7.2-RELEASE-p4 box doing a similar operation: > > =A0tin =A0tout =A0KB/t tps =A0MB/s =A0us ni sy in id > =A0 0 =A0 22 29.54 1208 34.86 =A0 3 =A00 56 =A02 39 > =A0 0 =A0 22 29.56 1177 33.97 =A0 3 =A00 57 =A01 39 > =A0 0 =A0 22 29.54 1201 34.64 =A0 3 =A00 58 =A02 37 > =A0 0 =A0 22 29.57 1144 33.04 =A0 2 =A00 51 =A03 44 > =A0 0 =A0 22 29.56 1126 32.52 =A0 3 =A00 54 =A02 42 > =A0 0 =A0 22 29.53 1179 34.01 =A0 3 =A00 53 =A02 42 > =A0 0 =A0 22 29.57 1165 33.65 =A0 2 =A00 58 =A02 38 > > One thing I notice is the larger block size the 7.2 writes but I don't > imagine that would be that significant. > > I've been using FreeBSD in various amateurish and wrong ways since > 2.2, so I wouldn't rule out me doing something stupid. =A0If so, I'd > love to know what. >